This is an automated email from the ASF dual-hosted git repository. chengpan pushed a commit to branch branch-1.5 in repository https://gitbox.apache.org/repos/asf/incubator-kyuubi.git
commit 5225b540952c23c5de67cc558217318b9169f1bb Author: jiaoqingbo <[email protected]> AuthorDate: Fri Mar 18 16:44:24 2022 +0800 [KYUUBI #2156][FOLLOWUP] Fix configuration format in document ### _Why are the changes needed?_ fix #2157 cofiguration format error ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #2165 from jiaoqingbo/2156-followup. Closes #2156 9862ddb2 [jiaoqingbo] [KYUUBI #2156][FOLLOWUP] Fix configuration format in document 47157c5c [jiaoqingbo] Merge branch 'master' into 2156-followup 9c89c081 [jiaoqingbo] [KYUUBI #2156][FOLLOWUP]fix cofiguration format 46084328 [jiaoqingbo] Merge branch 'master' into 2156 96a22e59 [jiaoqingbo] add link to doc dc11a214 [jiaoqingbo] [KYUUBI #2156] Change log to reflect exactly why getting token failed Authored-by: jiaoqingbo <[email protected]> Signed-off-by: Cheng Pan <[email protected]> --- docs/security/authentication.md | 20 ++++++++++---------- docs/security/hadoop_credentials_manager.md | 20 ++++++++++---------- docs/security/kinit.md | 8 ++++---- 3 files changed, 24 insertions(+), 24 deletions(-) diff --git a/docs/security/authentication.md b/docs/security/authentication.md index 7ca3780..2497804 100644 --- a/docs/security/authentication.md +++ b/docs/security/authentication.md @@ -36,16 +36,16 @@ In `$KYUUBI_HOME/conf/kyuubi-defaults.conf`, specify `kyuubi.authentication` to Key | Default | Meaning | Since --- | --- | --- | --- -kyuubi\.authentication|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>NONE</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Client authentication types.<ul> <li>NOSASL: raw transport.</li> <li>NONE: no authentication check.</li> <li>KERBEROS: Kerberos/GSSAPI authentication.</li> <li>LDAP: Lightweight Directory Access Protocol authentication.</li></ul></div>|<div style='width: 20pt'>1.0.0</div> +<code>kyuubi.authentication</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>NONE</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Client authentication types.<ul> <li>NOSASL: raw transport.</li> <li>NONE: no authentication check.</li> <li>KERBEROS: Kerberos/GSSAPI authentication.</li> <li>LDAP: Lightweight Directory Access Protocol authentication.</li></ul></div>|<div style='width: 20pt'>1.0.0</div> Key | Default | Meaning | Since --- | --- | --- | --- -kyuubi\.authentication|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>NONE</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Client authentication types.<ul> <li>NOSASL: raw transport.</li> <li>NONE: no authentication check.</li> <li>KERBEROS: Kerberos/GSSAPI authentication.</li> <li>LDAP: Lightweight Directory Access Protocol authentication.</li></ul></div>|<div style='width: 20pt'>1.0.0</div> -kyuubi\.authentication<br>\.ldap\.base\.dn|<div style='width: 80pt;word-wrap: break-word;white-space: normal'><undefined></div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>LDAP base DN.</div>|<div style='width: 20pt'>1.0.0</div> -kyuubi\.authentication<br>\.ldap\.domain|<div style='width: 80pt;word-wrap: break-word;white-space: normal'><undefined></div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>LDAP base DN.</div>|<div style='width: 20pt'>1.0.0</div> -kyuubi\.authentication<br>\.ldap\.url|<div style='width: 80pt;word-wrap: break-word;white-space: normal'><undefined></div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>SPACE character separated LDAP connection URL(s).</div>|<div style='width: 20pt'>1.0.0</div> -kyuubi\.authentication<br>\.sasl\.qop|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>auth</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Sasl QOP enable higher levels of protection for Kyuubi communication with clients.<ul> <li>auth - authentication only (default)</li> <li>auth-int - authentication plus integrity protection</li> <li>auth-conf - authentication plus integrity and confidentiality protection. This is applicable only if Kyuubi is [...] +<code>kyuubi.authentication</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>NONE</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Client authentication types.<ul> <li>NOSASL: raw transport.</li> <li>NONE: no authentication check.</li> <li>KERBEROS: Kerberos/GSSAPI authentication.</li> <li>LDAP: Lightweight Directory Access Protocol authentication.</li></ul></div>|<div style='width: 20pt'>1.0.0</div> +<code>kyuubi.authentication.ldap.base.dn</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'><undefined></div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>LDAP base DN.</div>|<div style='width: 20pt'>1.0.0</div> +<code>kyuubi.authentication.ldap.domain</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'><undefined></div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>LDAP base DN.</div>|<div style='width: 20pt'>1.0.0</div> +<code>kyuubi.authentication.ldap.url</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'><undefined></div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>SPACE character separated LDAP connection URL(s).</div>|<div style='width: 20pt'>1.0.0</div> +<code>kyuubi.authentication.sasl.qop</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>auth</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Sasl QOP enable higher levels of protection for Kyuubi communication with clients.<ul> <li>auth - authentication only (default)</li> <li>auth-int - authentication plus integrity protection</li> <li>auth-conf - authentication plus integrity and confidentiality protection. This is applicable only if Kyu [...] #### Using KERBEROS @@ -64,10 +64,10 @@ Following configurations also need to be set to enable KERBEROS authentication: Key | Default | Meaning | Since --- | --- | --- | --- -kyuubi\.kinit<br>\.principal|<div style='width: 80pt;word-wrap: break-word;white-space: normal'><undefined></div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Name of the Kerberos principal.</div>|<div style='width: 20pt'>1.0.0</div> -kyuubi\.kinit\.keytab|<div style='width: 80pt;word-wrap: break-word;white-space: normal'><undefined></div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Location of Kyuubi server's keytab.</div>|<div style='width: 20pt'>1.0.0</div> -kyuubi\.kinit\.interval|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>PT1H</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How often will Kyuubi server run `kinit -kt [keytab] [principal]` to renew the local Kerberos credentials cache</div>|<div style='width: 20pt'>1.0.0</div> -kyuubi\.kinit\.max<br>\.attempts|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>10</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How many times will `kinit` process retry</div>|<div style='width: 20pt'>1.0.0</div> +<code>kyuubi.kinit.principal</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'><undefined></div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Name of the Kerberos principal.</div>|<div style='width: 20pt'>1.0.0</div> +<code>kyuubi.kinit.keytab</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'><undefined></div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Location of Kyuubi server's keytab.</div>|<div style='width: 20pt'>1.0.0</div> +<code>kyuubi.kinit.interval</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>PT1H</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How often will Kyuubi server run `kinit -kt [keytab] [principal]` to renew the local Kerberos credentials cache</div>|<div style='width: 20pt'>1.0.0</div> +<code>kyuubi.kinit.max.attempts</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>10</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How many times will `kinit` process retry</div>|<div style='width: 20pt'>1.0.0</div> Please refer to [Kinit Auxiliary Service](kinit.html) to get configuration steps. diff --git a/docs/security/hadoop_credentials_manager.md b/docs/security/hadoop_credentials_manager.md index 6bc6be6..a68c75b 100644 --- a/docs/security/hadoop_credentials_manager.md +++ b/docs/security/hadoop_credentials_manager.md @@ -70,11 +70,11 @@ Hadoop Credentials Manager is initialized when Kyuubi server starts. Key | Default | Meaning | Type | Since --- | --- | --- | --- | --- -kyuubi\.credentials<br>\.hadoopfs\.enabled|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>true</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Whether to renew Hadoop filesystem delegation tokens</div>|<div style='width: 30pt'>boolean</div>|<div style='width: 20pt'>1.4.0</div> -kyuubi\.credentials<br>\.hadoopfs\.uris|<div style='width: 65pt;word-wrap: break-word;white-space: normal'></div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Extra Hadoop filesystem URIs for which to request delegation tokens. The filesystem that hosts fs.defaultFS does not need to be listed here.</div>|<div style='width: 30pt'>seq</div>|<div style='width: 20pt'>1.4.0</div> -kyuubi\.credentials<br>\.hive\.enabled|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>true</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Whether to renew Hive metastore delegation token</div>|<div style='width: 30pt'>boolean</div>|<div style='width: 20pt'>1.4.0</div> -kyuubi\.credentials<br>\.renewal\.interval|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>PT1H</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>How often Kyuubi renews one user's delegation tokens</div>|<div style='width: 30pt'>duration</div>|<div style='width: 20pt'>1.4.0</div> -kyuubi\.credentials<br>\.renewal\.retry\.wait|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>PT1M</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>How long to wait before retrying to fetch new credentials after a failure.</div>|<div style='width: 30pt'>duration</div>|<div style='width: 20pt'>1.4.0</div> +<code>kyuubi.credentials.hadoopfs.enabled</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>true</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Whether to renew Hadoop filesystem delegation tokens</div>|<div style='width: 30pt'>boolean</div>|<div style='width: 20pt'>1.4.0</div> +<code>kyuubi.credentials.hadoopfs.uris</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'></div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Extra Hadoop filesystem URIs for which to request delegation tokens. The filesystem that hosts fs.defaultFS does not need to be listed here.</div>|<div style='width: 30pt'>seq</div>|<div style='width: 20pt'>1.4.0</div> +<code>kyuubi.credentials.hive.enabled</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>true</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Whether to renew Hive metastore delegation token</div>|<div style='width: 30pt'>boolean</div>|<div style='width: 20pt'>1.4.0</div> +<code>kyuubi.credentials.renewal.interval</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>PT1H</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>How often Kyuubi renews one user's delegation tokens</div>|<div style='width: 30pt'>duration</div>|<div style='width: 20pt'>1.4.0</div> +<code>kyuubi.credentials.renewal.retry.wait</code>|<div style='width: 65pt;word-wrap: break-word;white-space: normal'>PT1M</div>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>How long to wait before retrying to fetch new credentials after a failure.</div>|<div style='width: 30pt'>duration</div>|<div style='width: 20pt'>1.4.0</div> ### Required Security Configs @@ -83,8 +83,8 @@ The necessary configurations for hdfs and hive to obtain delegation token are as Key | Meaning | value --- | --- | --- -<div style='width: 40pt;word-wrap: break-word;white-space: normal'>hadoop.security.authentication</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>Set the authentication for the cluster</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>kerberos</div> -<div style='width: 40pt;word-wrap: break-word;white-space: normal'>hive.metastore.uris</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>URI for client to contact metastore server</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>thrift://{metastoreHost}:{metastorePort}}</div> -<div style='width: 40pt;word-wrap: break-word;white-space: normal'>hive.metastore.sasl.enabled</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>If true, the metastore thrift interface will be secured with SASL.Clients must authenticate with Kerberos.</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>true</div> -<div style='width: 40pt;word-wrap: break-word;white-space: normal'>hive.metastore.kerberos.principal</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>The service principal for the metastore thrift server. The special string _HOST will be replaced automatically with the correct host name.</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>for example hive/_HOST@${realm}</div> -<div style='width: 40pt;word-wrap: break-word;white-space: normal'>hive.metastore.kerberos.keytab.file</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>The path to the Kerberos Keytab file containing the metastore thrift server's service principal.</div>|<div style='width: 40pt;word-wrap: break-word;white-space: normal'>for example /etc/security/keytabs/hive.service.keytab</div> \ No newline at end of file +<code>hadoop.security.authentication</code>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>Set the authentication for the cluster</div>|<div style='width: 120pt;word-wrap: break-word;white-space: normal'>kerberos</div> +<code>hive.metastore.uris</code>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>URI for client to contact metastore server</div>|<div style='width: 120pt;word-wrap: break-word;white-space: normal'>thrift://{metastoreHost}:{metastorePort}}</div> +<code>hive.metastore.sasl.enabled</code>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>If true, the metastore thrift interface will be secured with SASL.Clients must authenticate with Kerberos.</div>|<div style='width: 120pt;word-wrap: break-word;white-space: normal'>true</div> +<code>hive.metastore.kerberos.principal</code>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The service principal for the metastore thrift server. The special string _HOST will be replaced automatically with the correct host name.</div>|<div style='width: 120pt;word-wrap: break-word;white-space: normal'>for example hive/_HOST@${realm}</div> +<code>hive.metastore.kerberos.keytab.file</code>|<div style='width: 170pt;word-wrap: break-word;white-space: normal'>The path to the Kerberos Keytab file containing the metastore thrift server's service principal.</div>|<div style='width: 120pt;word-wrap: break-word;white-space: normal'>for example /etc/security/keytabs/hive.service.keytab</div> diff --git a/docs/security/kinit.md b/docs/security/kinit.md index 4cefef0..e78b77e 100644 --- a/docs/security/kinit.md +++ b/docs/security/kinit.md @@ -77,10 +77,10 @@ They are valid for relatively short period. So, we always need to refresh it for Key | Default | Meaning | Since --- | --- | --- | --- -kyuubi\.kinit<br>\.principal|<div style='width: 80pt;word-wrap: break-word;white-space: normal'><undefined></div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Name of the Kerberos principal.</div>|<div style='width: 20pt'>1.0.0</div> -kyuubi\.kinit\.keytab|<div style='width: 80pt;word-wrap: break-word;white-space: normal'><undefined></div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Location of Kyuubi server's keytab.</div>|<div style='width: 20pt'>1.0.0</div> -kyuubi\.kinit\.interval|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>PT1H</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How often will Kyuubi server run `kinit -kt [keytab] [principal]` to renew the local Kerberos credentials cache</div>|<div style='width: 20pt'>1.0.0</div> -kyuubi\.kinit\.max<br>\.attempts|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>10</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How many times will `kinit` process retry</div>|<div style='width: 20pt'>1.0.0</div> +<code>kyuubi.kinit.principal</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'><undefined></div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Name of the Kerberos principal.</div>|<div style='width: 20pt'>1.0.0</div> +<code>kyuubi.kinit.keytab</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'><undefined></div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>Location of Kyuubi server's keytab.</div>|<div style='width: 20pt'>1.0.0</div> +<code>kyuubi.kinit.interval</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>PT1H</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How often will Kyuubi server run `kinit -kt [keytab] [principal]` to renew the local Kerberos credentials cache</div>|<div style='width: 20pt'>1.0.0</div> +<code>kyuubi.kinit.max.attempts</code>|<div style='width: 80pt;word-wrap: break-word;white-space: normal'>10</div>|<div style='width: 200pt;word-wrap: break-word;white-space: normal'>How many times will `kinit` process retry</div>|<div style='width: 20pt'>1.0.0</div> When working with a Kerberos-enabled Hadoop cluster, we should ensure that `hadoop.security.authentication` is set to `KERBEROS` in `$HADOOP_CONF_DIR/core-site.xml` or `$KYUUBI_HOME/conf/kyuubi-defaults.conf`.
