[
https://issues.apache.org/jira/browse/HDDS-1019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777061#comment-16777061
]
Elek, Marton commented on HDDS-1019:
------------------------------------
I tested it with the secure docker-compose and I have some more suggestions:
1. starter script runs with 'set \-e'. We need a set \+e/set -e for checking
the availability of the KDC service:
{code}
while true
do
set +e
STATUS=$(curl -s -o /dev/null -w '%{http_code}'
http://"$ISSUER_SERVER"/keytab/test/test)
set -e
if [ "$STATUS" -eq 200 ]; then
echo "Got 200, KDC service ready!!"
break
else
echo "Got $STATUS :( KDC service not ready yet..."
fi
sleep 5
done
{code}
Without this modification the script is stopped if the kdc is not avilable.
2. $CONF_DIR is confusing (for me). I would use something like $KEYTAB_DIR
instead. And I think the default could be /etc/security/keytabs (Now we have a
hard dependency that the $CONF_DIR should be set for a secure environment. I
think it's better to use a default value in starter.sh)
3. docker-image/runner can be deleted from the trunk
4. I would use the apache/hadoop-runner as a base image for the krb5 image to
use exactly the same mit kerberos (I noticed an error that the keytab versions
are different before this change).
Here is my proposed Dockerfile:
{code}
FROM apache/hadoop-runner
USER root
WORKDIR /opt
RUN yum install -y krb5-server
RUN curl -L
https://github.com/flokkr/issuer/releases/download/1.0.2/issuer_1.0.2_linux_amd64.tar.gz
| tar xzv
ADD issuer /opt/
RUN chmod +x /opt/issuer
ADD krb5.conf /etc/
ADD kadm5.acl /var/lib/krb5kdc/kadm5.acl
RUN kdb5_util create -s -P Welcome1
RUN kadmin.local -q "addprinc -randkey admin/[email protected]"
RUN kadmin.local -q "ktadd -k /tmp/admin.keytab admin/[email protected]"
RUN kadmin.local -q "add_policy -maxlife 7d default"
ADD launcher.sh /opt/
RUN chmod +x /opt/launcher.sh
RUN mkdir -p /data
ENTRYPOINT ["/usr/local/bin/dumb-init", "--", "/opt/launcher.sh"]
{code}
Note 1: I added a default policy to avoid a warning (add_policy)
Note 2: I updated the issuer version to 1.0.2. This version waits until the kdc
is available (which is started in the background) which is more safe.
5. For centos the max_renewable_life is required in the krb5.conf.
{code}
[realms]
EXAMPLE.COM = {
kdc = localhost
admin_server = localhost
max_renewable_life = 7d
}
{code}
Without this change we can get a cryptic error (with java11 but not with java8):
{code}
2019-02-25 12:44:07 INFO StorageContainerManager:51 - registered UNIX signal
handlers for [TERM, HUP, INT]
2019-02-25 12:44:08 WARN ServerUtils:147 - ozone.scm.db.dirs is not
configured. We recommend adding this setting. Falling back to
ozone.metadata.dirs instead.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by
org.apache.hadoop.security.authentication.util.KerberosUtil
(file:/opt/hadoop/share/ozone/lib/hadoop-auth-3.2.0.jar) to method
sun.security.krb5.Config.getInstance()
WARNING: Please consider reporting this to the maintainers of
org.apache.hadoop.security.authentication.util.KerberosUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal
reflective access operations
WARNING: All illegal access operations will be denied in a future release
2019-02-25 12:44:08 ERROR StorageContainerManager:594 - Failed to start the
StorageContainerManager.
org.apache.hadoop.security.KerberosAuthException: failure to login: for
principal: scm/[email protected] from keytab /etc/security/keytabs/scm.keytab
javax.security.auth.login.LoginException: Message stream modified (41)
at
org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1847)
at
org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1215)
at
org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1008)
at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:315)
at
org.apache.hadoop.hdds.scm.server.StorageContainerManager.loginAsSCMUser(StorageContainerManager.java:494)
at
org.apache.hadoop.hdds.scm.server.StorageContainerManager.initializeCAnSecurityProtocol(StorageContainerManager.java:442)
at
org.apache.hadoop.hdds.scm.server.StorageContainerManager.<init>(StorageContainerManager.java:260)
at
org.apache.hadoop.hdds.scm.server.StorageContainerManager.<init>(StorageContainerManager.java:216)
at
org.apache.hadoop.hdds.scm.server.StorageContainerManager.createSCM(StorageContainerManager.java:675)
at
org.apache.hadoop.hdds.scm.server.StorageContainerManager.main(StorageContainerManager.java:587)
Caused by: javax.security.auth.login.LoginException: Message stream modified
(41)
at
jdk.security.auth/com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:781)
at
jdk.security.auth/com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:592)
at
java.base/javax.security.auth.login.LoginContext.invoke(LoginContext.java:726)
at
java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:665)
at
java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:663)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at
java.base/javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:663)
at
java.base/javax.security.auth.login.LoginContext.login(LoginContext.java:574)
at
org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:1926)
at
org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1837)
... 9 more
Caused by: KrbException: Message stream modified (41)
at java.security.jgss/sun.security.krb5.KrbKdcRep.check(KrbKdcRep.java:83)
at java.security.jgss/sun.security.krb5.KrbAsRep.decrypt(KrbAsRep.java:158)
at
java.security.jgss/sun.security.krb5.KrbAsRep.decryptUsingKeyTab(KrbAsRep.java:121)
at
java.security.jgss/sun.security.krb5.KrbAsReqBuilder.resolve(KrbAsReqBuilder.java:295)
at
java.security.jgss/sun.security.krb5.KrbAsReqBuilder.action(KrbAsReqBuilder.java:371)
at
jdk.security.auth/com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:753)
... 18 more
2019-02-25 12:44:08 INFO ExitUtil:210 - Exiting with status 1:
org.apache.hadoop.security.KerberosAuthException: failure to login: for
principal: scm/[email protected] from keytab /etc/security/keytabs/scm.keytab
javax.security.auth.login.LoginException: Message stream modified (41)
{code}
> Use apache/hadoop-runner image to test ozone secure cluster
> -----------------------------------------------------------
>
> Key: HDDS-1019
> URL: https://issues.apache.org/jira/browse/HDDS-1019
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Reporter: Elek, Marton
> Assignee: Xiaoyu Yao
> Priority: Critical
> Attachments: HDDS-1019-docker-hadoop-runner.01.patch,
> HDDS-1019-docker-hadoop-runner.02.patch, HDDS-1019-trunk.01.patch
>
>
> As of now the secure ozone cluster uses a custom image which is not based on
> the apache/hadoop-runner image. There are multiple problems with that:
> 1. multiple script files which are maintained in the docker-hadoop-runner
> branch are copied and duplicated in
> hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/runner/scripts
> 2. The user of the image is root. It creates
> core-site.xml/hdfs-site.xml/ozone-site.xml which root user which prevents to
> run all the default smoke tests
> 3. To build the base image with each build takes more time
> I propose to check what is missing from the apache/hadoop-ozone base image,
> add it and use that one.
> I marked it critical because 2): it breaks the run of the the acceptance test
> suit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
