[
https://issues.apache.org/jira/browse/FLINK-8981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16550699#comment-16550699
]
ASF GitHub Bot commented on FLINK-8981:
---------------------------------------
Github user dawidwys commented on a diff in the pull request:
https://github.com/apache/flink/pull/6377#discussion_r204020749
--- Diff:
flink-end-to-end-tests/test-scripts/docker-hadoop-secure-cluster/README.md ---
@@ -0,0 +1,118 @@
+# Apache Hadoop Docker image with Kerberos enabled
+
+This image is modified version of Knappek/docker-hadoop-secure
+ * Knappek/docker-hadoop-secure
<https://github.com/Knappek/docker-hadoop-secure>
+
+With bits and pieces added from Lewuathe/docker-hadoop-cluster to extend
it to start a proper kerberized Hadoop cluster:
+ * Lewuathe/docker-hadoop-cluster
<https://github.com/Lewuathe/docker-hadoop-cluster>
+
+And a lot of added stuff for making this an actual, properly configured,
kerberized cluster with proper user/permissions structure.
+
+Versions
+--------
+
+* JDK8
+* Hadoop 2.8.3
+
+Default Environment Variables
+-----------------------------
+
+| Name | Value | Description |
+| ---- | ---- | ---- |
+| `KRB_REALM` | `EXAMPLE.COM` | The Kerberos Realm, more information
[here](https://web.mit.edu/kerberos/krb5-1.12/doc/admin/conf_files/krb5_conf.html#)
|
+| `DOMAIN_REALM` | `example.com` | The Kerberos Domain Realm, more
information
[here](https://web.mit.edu/kerberos/krb5-1.12/doc/admin/conf_files/krb5_conf.html#)
|
+| `KERBEROS_ADMIN` | `admin/admin` | The KDC admin user |
+| `KERBEROS_ADMIN_PASSWORD` | `admin` | The KDC admin password |
+
+You can simply define these variables in the `docker-compose.yml`.
+
+Run image
+---------
+
+Clone the [Github
project](https://github.com/aljoscha/docker-hadoop-secure-cluster) and run
+
+```
+docker-compose up
+```
+
+Usage
+-----
+
+Get the container name with `docker ps` and login to the container with
+
+```
+docker exec -it <container-name> /bin/bash
+```
+
+
+To obtain a Kerberos ticket, execute
+
+```
+kinit -kt /home/hadoop-user/hadoop-user.keytab hadoop-user
+```
+
+Afterwards you can use `hdfs` CLI like
+
+```
+hdfs dfs -ls /
+```
+
+
+Known issues
+------------
+
+### Unable to obtain Kerberos password
+
+#### Error
+docker-compose up fails for the first time with the error
+
+```
+Login failure for nn/[email protected] from keytab
/etc/security/keytabs/nn.service.keytab:
javax.security.auth.login.LoginException: Unable to obtain password from user
+```
+
+#### Solution
+
+Stop the containers with `docker-compose down` and start again with
`docker-compose up -d`.
+
+
+### JDK 8
+
+Make sure you use download a JDK version that is still available. Old
versions can be deprecated by Oracle and thus the download link won't be able
anymore.
+
+Get the latest JDK8 Download URL with
+
+```
+curl -s https://lv.binarybabel.org/catalog-api/java/jdk8.json
+```
+
+### Java Keystore
+
+If the Keystroe has been expired, then create a new `keystore.jks`:
--- End diff --
Keystroe -> Keystore
Won't it be a problem in tests? Will the test start failing one day because
of the keystore expired?
> Add end-to-end test for running on YARN with Kerberos
> -----------------------------------------------------
>
> Key: FLINK-8981
> URL: https://issues.apache.org/jira/browse/FLINK-8981
> Project: Flink
> Issue Type: Sub-task
> Components: Security, Tests
> Affects Versions: 1.5.0
> Reporter: Till Rohrmann
> Assignee: Aljoscha Krettek
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.6.0
>
>
> We should add an end-to-end test which verifies Flink's integration with
> Kerberos security. In order to do this, we should start a Kerberos secured
> Hadoop, ZooKeeper and Kafka cluster. Then we should start a Flink cluster
> with HA enabled and run a job which reads from and writes to Kafka. We could
> use a simple pipe job for that purpose which has some state for checkpointing
> to HDFS.
> See [security docs|
> https://ci.apache.org/projects/flink/flink-docs-master/ops/security-kerberos.html]
> for how more information about Flink's Kerberos integration.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
