On 03/15/2012 06:40 PM, Harsh J (Created) (JIRA) wrote:
Consider splitting homedir between mapred and hdfs users?
---------------------------------------------------------
Key: BIGTOP-456
URL: https://issues.apache.org/jira/browse/BIGTOP-456
Project: Bigtop
Issue Type: Improvement
Components: General
Affects Versions: 0.1.0
Environment: RPMs
Reporter: Harsh J
Priority: Minor
Both "mapred" and "hdfs" users have the same home dir.
A user reported having some problems with their config management system overwriting the "mapred"
user permissions of the PID directory (Which is also its homedir) with those of the "hdfs" user
(Same homedir as "mapred" user), which causes the tasktracker process to fail to start, since it
now cannot write to the PID dir.
Although the config system can be fixed not to do that, if both users had
separate home dirs, this would not have been a problem, and the separation
would have only been logical.
I think after the username separation Hadoop has had in packaging terms, the
homedir split does make sense.
Its just 1/0.22 versions of Hadoop and their packages that could be affected by
this.
Presently, for 0.23+, I think we have /var/run/hadoop/ for all things HDFS
(Should we rename?) and /var/run/yarn/ for all things MapReduce2 which makes
sense and should be good enough.
Yes, the /var/run/hadoop directory should be the right for it, although
on the new versions of Fedora (15 or more), the default directory was
changed to /run, so I don't know which is the best approach for it.
Fedora 15 Releases Notes
3. Changes for System Administrators
3.2.2 /run directory
"Fedora 15 has a |/run| directory for storing runtime data. |/run| is
now a tmpfs, and |/var/run| is |bind| mounted to it. |/var/lock| is
|bind| mounted to |/run/lock|. Applications can use |/run| the same way
as |/var/run|. Several programs including |udev|, |dracut|, |mdadm|,
|mount| and |initscripts| used hidden directories under |/dev| for
runtime data during early bootup before |/var| is mounted. However
|/dev/| is supposed to be used for only device nodes and there is
consensus between major distributions to shift to using |/run| instead.
Fedora 15 is leading this change. Details including the benefits are
explained here
<http://lists.fedoraproject.org/pipermail/devel/2011-March/150031.html>.
This change /is/ compliant with the Filesystem Hierarchy Standard
<http://www.pathname.com/fhs/pub/fhs-2.3.html#THEROOTFILESYSTEM>, which
allows distributions to create new directories in the root hierarchy as
long as there is careful consideration of the consequences. Co-author of
the latest FHS specification has expressed support
<https://lwn.net/Articles/436177/> for this change. Lennart Poettering
has filed a request <http://bugs.freestandards.org/show_bug.cgi?id=718>
to update the FHS standard to include this change as well."
3.2.3 /var/run/ and /var/lock
|/var/run| and |/var/lock| are now bind mounted to |/run| and
|/run/lock| from tmpfs, and hence emptied on reboot. Applications must
ensure to recreate their own files/dirs on startup, and cannot rely that
doing this at package installation will suffice. It is possible to use
systemd's |tmpfiles.d| mechanism to recreate directories and files
beneath |/var/run| and |/var/lock| on boot, if necessary. See
|tmpfiles.d(5)| for details
(http://0pointer.de/public/systemd-man/tmpfiles.d.html) and the conf
files in |/etc/tmpfiles.d| for examples of such configuration. Fedora
packaging guidelines for |tmpfiles.d| is at
http://fedoraproject.org/wiki/Packaging:Tmpfiles.d.
http://docs.fedoraproject.org/en-US/Fedora/15/html/Release_Notes/sect-Release_Notes-Changes_for_SysAdmin.html
Regards
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS
INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci
--
Marcos Luis OrtÃz Valmaseda (@marcosluis2186)
Data Engineer at UCI
http://marcosluis2186.posterous.com
10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS
INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci
