Thanks, Rekha, This information is useful for me.
I have another question. Since I am using Debian 32-bit Linux, I need the 32-bit binary file taskcontroller. However, I found the binary files provided in hadoop 1.0.3 is 64 bit. I downloaded the hadoop build file from server jenkins (https://builds.apache.org/job/Hadoop-1.0-Build/ws/trunk/build/c++-build/Linux-i386-32/task-controller/). It's still a 64 bit file. I got the following errors when I start task tracker using the hadoop 64-bit taskcontroller: 12/09/17 11:59:58 ERROR mapred.TaskTracker: Can not start task tracker because java.io.IOException: Task controller setup failed because of invalidpermissions/ownership with exit code 126 at org.apache.hadoop.mapred.LinuxTaskController.setup(LinuxTaskController.java:143) at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1452) at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3742) Caused by: org.apache.hadoop.util.Shell$ExitCodeException: /opt/ywang/hadoop-1.0.3/libexec/../bin/task-controller: /opt/ywang/hadoop-1.0.3/libexec/../bin/task-controller: cannot execute binary file at org.apache.hadoop.util.Shell.runCommand(Shell.java:255) at org.apache.hadoop.util.Shell.run(Shell.java:182) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375) at org.apache.hadoop.mapred.LinuxTaskController.setup(LinuxTaskController.java:137) I am wondering if not providing 32-bit of taskcontroller is a build bug, or 64-bit taskcontroller can be used somehow on the 32-bit platform? If no 32-bit executable is provided in the daily build of hadoop, how can I build one by myself? Thanks! Yongzhi On Mon, Sep 17, 2012 at 5:42 AM, Joshi, Rekha <[email protected]> wrote: > Hi Yongzhi, > > Well, I don't know if this will help, but I looked into source code, can > see all token, authentication related features discussed in the design > under- o.a.h.hdfs.security.*, o.a.h.mapreduce.security.*, o.a.h.security.* > , o.a.h.security.authentication.* > And HADOOP-4487 is marked fixed now, so there might be explicit bug issue, > but features are in. > Comparing the release notes can also give more details - > http://hadoop.apache.org/docs/r1.0.3/releasenotes.html with > http://hadoop.apache.org/docs/r1.0.0/releasenotes.html > > Owen session on security is good, albeit a bit old - > http://developer.yahoo.com/blogs/ydn/posts/2010/07/hadoop_security_in_detai > l/ > For kerberos itself, this is neat - > http://www.ornl.gov/~jar/HowToKerb.html and > http://www.cmf.nrl.navy.mil/krb/kerberos-faq.html > > So installing kerberos itself would be almost similar steps across CDH4, > Hortonworks , Yahoo! - only configuration would need to be correctly setup > in kerberos.principal, authentication.type in core-site.xml > Some more examples - > http://hortonworks.com/blog/fine-tune-your-apache-hadoop-security-settings/ > #more-1124 > https://cwiki.apache.org/GIRAPH/quick-start-running-giraph-with-secure-hado > op.html > > Thanks > Rekha > > > > On 16/09/12 8:57 AM, "Yongzhi Wang" <[email protected]> wrote: > >>Dear All, >> >>I am confused about the usage of Kerberos on Hadoop 1.0.3. >> >>I have difficulty in finding some documents to configure of the >>security feature of HADOOP 1.0.3. Specifically, how should I configure >>the Hadoop, so that I can use Kerberos? The only document that is >>related with this question is CDH4 Security Guide >>(https://ccp.cloudera.com/display/CDH4DOC/CDH4+Security+Guide), an >>instruction about the security configuration for CloudEra Distributed >>Hadoop. But I am not sure if this guide can be directly used to >>configure the Apache Hadoop 1.0.3. Afterall, I don't know how many >>differences exist between the CDH4 and Apache Hadoop 1.0.3. >> >>I read some materials published by the hadoop development team, >>including the documentation posted on the apache website >>(http://hadoop.apache.org/docs/r1.0.3/index.html) and the "Hadoop >>Security Design" document proposed by Yahoo! in 2009. Unfortunately, I >>still can not generate a clear vision after I read those documents. >>All my questions are derived from one basic question: Are all of the >>design features in "Hadoop Security Design" included in the release >>1.0.3? If not, which of those features are introduced in release >>1.0.3? Which features are included in the Hadoop 2.0? Which features >>are still not implemented? >> >>For example, the "Hadoop Security Design" document mentioned three >>types of tokens (Delegation Token, Block Access Token and Job Token). >>Did release 1.0.3 support all the three types of tokens? >> >>In the 1.0.3 document "hdfs permission guide" >>(http://hadoop.apache.org/docs/r1.0.3/hdfs_permissions_guide.html), it >>mentions that "In this release of Hadoop the identity of a client >>process is just whatever the host operating system says it is. For >>Unix-like systems, ......In the future there will be other ways of >>establishing user identity (think Kerberos, LDAP, and others). >>......". It seems the 1.0.3 does not fully support Kerberos. If in >>that case, to what degree does the release 1.0.3 support Kerberos? >> >>So my question is: >> >> 1. Is there any document comparing the security feature in each >>release of hadoop with the "Hadoop Security Design" proposed by Yahoo! >>? >> 2. In release 1.0.3, which component of hadoop can use Kerberos to >>leverage security? In order to use the Kerberos, how should I >>configure Hadoop? >> >>I am not very familiar with Kerberos. So if I have some >>misunderstanding, please feel free to point out. >> >>Thanks! >> >>Best regards, >>Yongzhi >
