This is an automated email from the ASF dual-hosted git repository.
bbende pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/master by this push:
new 35d1cac Documentation : ConsumerKafka_2_0 - updated configuration
details re… (#3360)
35d1cac is described below
commit 35d1cace080a476910b007e67ce4e90661a0d299
Author: pushpavanthar <[email protected]>
AuthorDate: Wed Mar 13 20:42:00 2019 +0530
Documentation : ConsumerKafka_2_0 - updated configuration details re…
(#3360)
* Documentation : ConsumerKafka_2_0 - updated configuration details
related to SASL_PLAINTEXT - SCRAM
* Updated rest of the related documents with SCRAM configurations and few
NOTEs
* incorporated review comments
* incorporated review comments
---
.../additionalDetails.html | 74 ++++++++++++++++++++--
.../additionalDetails.html | 74 ++++++++++++++++++++--
.../additionalDetails.html | 49 ++++++++++++++
.../additionalDetails.html | 49 ++++++++++++++
4 files changed, 234 insertions(+), 12 deletions(-)
diff --git
a/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-0-processors/src/main/resources/docs/org.apache.nifi.processors.kafka.pubsub.ConsumeKafkaRecord_2_0/additionalDetails.html
b/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-0-processors/src/main/resources/docs/org.apache.nifi.processors.kafka.pubsub.ConsumeKafkaRecord_2_0/additionalDetails.html
index 98191b7..8f7c4e9 100644
---
a/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-0-processors/src/main/resources/docs/org.apache.nifi.processors.kafka.pubsub.ConsumeKafkaRecord_2_0/additionalDetails.html
+++
b/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-0-processors/src/main/resources/docs/org.apache.nifi.processors.kafka.pubsub.ConsumeKafkaRecord_2_0/additionalDetails.html
@@ -73,12 +73,7 @@
</p>
<h4>SASL_PLAINTEXT - GSSAPI</h4>
<p>
- If the SASL mechanism is GSSAPI, then the client must provide a
JAAS configuration to authenticate. The
- JAAS configuration can be provided by specifying the
java.security.auth.login.config system property in
- NiFi's bootstrap.conf, such as:
- <pre>
-
java.arg.16=-Djava.security.auth.login.config=/path/to/kafka_client_jaas.conf
- </pre>
+ If the SASL mechanism is GSSAPI, then the client must provide a
JAAS configuration to authenticate.
</p>
<p>
An example of the JAAS config file would be the following:
@@ -95,6 +90,25 @@
<b>NOTE:</b> The serviceName in the JAAS file must match the Kerberos
Service Name in the processor.
</p>
<p>
+ The JAAS configuration can be provided by either of below ways
+ <ol type="1">
+ <li>specify the java.security.auth.login.config system property in
+ NiFi's bootstrap.conf. This limits you to use only one user
credential across the cluster.</li>
+ <pre>
+
java.arg.16=-Djava.security.auth.login.config=/path/to/kafka_client_jaas.conf
+ </pre>
+ <li>add user attribute 'sasl.jaas.config' in the processor
configurations. This method allows one to have multiple consumers with
different user credentials or gives flexibility to consume from multiple kafka
clusters.</li>
+ <pre>
+ sasl.jaas.config :
com.sun.security.auth.module.Krb5LoginModule required
+ useKeyTab=true
+ storeKey=true
+ keyTab="/path/to/nifi.keytab"
+ serviceName="kafka"
+ principal="[email protected]";
+ </pre>
+ </ol>
+ </p>
+ <p>
Alternatively, the JAAS
configuration when using GSSAPI can be provided by specifying the
Kerberos Principal and Kerberos Keytab
directly in the processor properties. This will dynamically create
a JAAS configuration like above, and
@@ -112,16 +126,64 @@
password="nifi-password";
};
</pre>
+ The JAAS configuration can be provided by either of below ways
+ <ol type="1">
+ <li>specify the java.security.auth.login.config system property in
+ NiFi's bootstrap.conf. This limits you to use only one user
credential across the cluster.</li>
+ <pre>
+
java.arg.16=-Djava.security.auth.login.config=/path/to/kafka_client_jaas.conf
+ </pre>
+ <li>add user attribute 'sasl.jaas.config' in the processor
configurations. This method allows one to have multiple consumers with
different user credentials or gives flexibility to consume from multiple kafka
clusters.</li>
+ <pre>
+ sasl.jaas.config :
org.apache.kafka.common.security.plain.PlainLoginModule required
+ username="nifi"
+ password="nifi-password";
+ </pre>
+ <b>NOTE:</b> The dynamic properties of this processor are not
secured and as a result the password entered when utilizing sasl.jaas.config
will be stored in the flow.xml.gz file in plain-text, and will be saved to NiFi
Registry if using versioned flows.
+ </ol>
</p>
<p>
<b>NOTE:</b> It is not recommended to use a SASL mechanism of
PLAIN with SASL_PLAINTEXT, as it would transmit
the username and password unencrypted.
</p>
<p>
+ <b>NOTE:</b> The Kerberos Service Name is not required for SASL
mechanism of PLAIN. However, processor warns saying this attribute has to be
filled with non empty string. You can choose to fill any random string, such as
"null".
+ </p>
+ <p>
<b>NOTE:</b> Using the PlainLoginModule will cause it be
registered in the JVM's static list of Providers, making
it visible to components in other NARs that may access the
providers. There is currently a known issue
where Kafka processors using the PlainLoginModule will cause HDFS
processors with Keberos to no longer work.
</p>
+ <h4>SASL_PLAINTEXT - SCRAM</h4>
+ <p>
+ If the SASL mechanism is SCRAM, then client must provide a JAAS
configuration to authenticate, but
+ the JAAS configuration must use Kafka's ScramLoginModule. Ensure
that you add user defined attribute 'sasl.mechanism' and assign 'SCRAM-SHA-256'
or 'SCRAM-SHA-512' based on kafka broker configurations. An example of the JAAS
config file would
+ be the following:
+ <pre>
+ KafkaClient {
+ org.apache.kafka.common.security.scram.ScramLoginModule required
+ username="nifi"
+ password="nifi-password";
+ };
+ </pre>
+ The JAAS configuration can be provided by either of below ways
+ <ol type="1">
+ <li>specify the java.security.auth.login.config system property in
+ NiFi's bootstrap.conf. This limits you to use only one user
credential across the cluster.</li>
+ <pre>
+
java.arg.16=-Djava.security.auth.login.config=/path/to/kafka_client_jaas.conf
+ </pre>
+ <li>add user attribute 'sasl.jaas.config' in the processor
configurations. This method allows one to have multiple consumers with
different user credentials or gives flexibility to consume from multiple kafka
clusters.</li>
+ <pre>
+ sasl.jaas.config :
org.apache.kafka.common.security.scram.ScramLoginModule required
+ username="nifi"
+ password="nifi-password";
+ </pre>
+ <b>NOTE:</b> The dynamic properties of this processor are not
secured and as a result the password entered when utilizing sasl.jaas.config
will be stored in the flow.xml.gz file in plain-text, and will be saved to NiFi
Registry if using versioned flows.
+ </ol>
+ <p>
+ <b>NOTE:</b> The Kerberos Service Name is not required for SASL
mechanism of SCRAM-SHA-256 or SCRAM-SHA-512. However, processor warns saying
this attribute has to be filled with non empty string. You can choose to fill
any random string, such as "null".
+ </p>
<h3>SASL_SSL</h3>
<p>
This option uses SASL with an SSL/TLS transport layer to
authenticate to the broker. In order to use this
diff --git
a/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-0-processors/src/main/resources/docs/org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_2_0/additionalDetails.html
b/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-0-processors/src/main/resources/docs/org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_2_0/additionalDetails.html
index b6525cb..bd061b2 100644
---
a/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-0-processors/src/main/resources/docs/org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_2_0/additionalDetails.html
+++
b/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-0-processors/src/main/resources/docs/org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_2_0/additionalDetails.html
@@ -73,12 +73,7 @@
</p>
<h4>SASL_PLAINTEXT - GSSAPI</h4>
<p>
- If the SASL mechanism is GSSAPI, then the client must provide a
JAAS configuration to authenticate. The
- JAAS configuration can be provided by specifying the
java.security.auth.login.config system property in
- NiFi's bootstrap.conf, such as:
- <pre>
-
java.arg.16=-Djava.security.auth.login.config=/path/to/kafka_client_jaas.conf
- </pre>
+ If the SASL mechanism is GSSAPI, then the client must provide a
JAAS configuration to authenticate.
</p>
<p>
An example of the JAAS config file would be the following:
@@ -95,6 +90,25 @@
<b>NOTE:</b> The serviceName in the JAAS file must match the Kerberos
Service Name in the processor.
</p>
<p>
+ The JAAS configuration can be provided by either of below ways
+ <ol type="1">
+ <li>specify the java.security.auth.login.config system property in
+ NiFi's bootstrap.conf. This limits you to use only one user
credential across the cluster.</li>
+ <pre>
+
java.arg.16=-Djava.security.auth.login.config=/path/to/kafka_client_jaas.conf
+ </pre>
+ <li>add user attribute 'sasl.jaas.config' in the processor
configurations. This method allows one to have multiple consumers with
different user credentials or gives flexibility to consume from multiple kafka
clusters.</li>
+ <pre>
+ sasl.jaas.config :
com.sun.security.auth.module.Krb5LoginModule required
+ useKeyTab=true
+ storeKey=true
+ keyTab="/path/to/nifi.keytab"
+ serviceName="kafka"
+ principal="[email protected]";
+ </pre>
+ </ol>
+ </p>
+ <p>
Alternatively, the JAAS
configuration when using GSSAPI can be provided by specifying the
Kerberos Principal and Kerberos Keytab
directly in the processor properties. This will dynamically create
a JAAS configuration like above, and
@@ -112,16 +126,64 @@
password="nifi-password";
};
</pre>
+ The JAAS configuration can be provided by either of below ways
+ <ol type="1">
+ <li>specify the java.security.auth.login.config system property in
+ NiFi's bootstrap.conf. This limits you to use only one user
credential across the cluster.</li>
+ <pre>
+
java.arg.16=-Djava.security.auth.login.config=/path/to/kafka_client_jaas.conf
+ </pre>
+ <li>add user attribute 'sasl.jaas.config' in the processor
configurations. This method allows one to have multiple consumers with
different user credentials or gives flexibility to consume from multiple kafka
clusters.</li>
+ <pre>
+ sasl.jaas.config :
org.apache.kafka.common.security.plain.PlainLoginModule required
+ username="nifi"
+ password="nifi-password";
+ </pre>
+ <b>NOTE:</b> The dynamic properties of this processor are not
secured and as a result the password entered when utilizing sasl.jaas.config
will be stored in the flow.xml.gz file in plain-text, and will be saved to NiFi
Registry if using versioned flows.
+ </ol>
</p>
<p>
<b>NOTE:</b> It is not recommended to use a SASL mechanism of
PLAIN with SASL_PLAINTEXT, as it would transmit
the username and password unencrypted.
</p>
<p>
+ <b>NOTE:</b> The Kerberos Service Name is not required for SASL
mechanism of PLAIN. However, processor warns saying this attribute has to be
filled with non empty string. You can choose to fill any random string, such as
"null".
+ </p>
+ <p>
<b>NOTE:</b> Using the PlainLoginModule will cause it be
registered in the JVM's static list of Providers, making
it visible to components in other NARs that may access the
providers. There is currently a known issue
where Kafka processors using the PlainLoginModule will cause HDFS
processors with Keberos to no longer work.
</p>
+ <h4>SASL_PLAINTEXT - SCRAM</h4>
+ <p>
+ If the SASL mechanism is SCRAM, then client must provide a JAAS
configuration to authenticate, but
+ the JAAS configuration must use Kafka's ScramLoginModule. Ensure
that you add user defined attribute 'sasl.mechanism' and assign 'SCRAM-SHA-256'
or 'SCRAM-SHA-512' based on kafka broker configurations. An example of the JAAS
config file would
+ be the following:
+ <pre>
+ KafkaClient {
+ org.apache.kafka.common.security.scram.ScramLoginModule required
+ username="nifi"
+ password="nifi-password";
+ };
+ </pre>
+ The JAAS configuration can be provided by either of below ways
+ <ol type="1">
+ <li>specify the java.security.auth.login.config system property in
+ NiFi's bootstrap.conf. This limits you to use only one user
credential across the cluster.</li>
+ <pre>
+
java.arg.16=-Djava.security.auth.login.config=/path/to/kafka_client_jaas.conf
+ </pre>
+ <li>add user attribute 'sasl.jaas.config' in the processor
configurations. This method allows one to have multiple consumers with
different user credentials or gives flexibility to consume from multiple kafka
clusters.</li>
+ <pre>
+ sasl.jaas.config :
org.apache.kafka.common.security.scram.ScramLoginModule required
+ username="nifi"
+ password="nifi-password";
+ </pre>
+ <b>NOTE:</b> The dynamic properties of this processor are not
secured and as a result the password entered when utilizing sasl.jaas.config
will be stored in the flow.xml.gz file in plain-text, and will be saved to NiFi
Registry if using versioned flows.
+ </ol>
+ <p>
+ <b>NOTE:</b> The Kerberos Service Name is not required for SASL
mechanism of SCRAM-SHA-256 or SCRAM-SHA-512. However, processor warns saying
this attribute has to be filled with non empty string. You can choose to fill
any random string, such as "null".
+ </p>
<h3>SASL_SSL</h3>
<p>
This option uses SASL with an SSL/TLS transport layer to
authenticate to the broker. In order to use this
diff --git
a/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-0-processors/src/main/resources/docs/org.apache.nifi.processors.kafka.pubsub.PublishKafkaRecord_2_0/additionalDetails.html
b/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-0-processors/src/main/resources/docs/org.apache.nifi.processors.kafka.pubsub.PublishKafkaRecord_2_0/additionalDetails.html
index dd89164..fc9a6e8 100644
---
a/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-0-processors/src/main/resources/docs/org.apache.nifi.processors.kafka.pubsub.PublishKafkaRecord_2_0/additionalDetails.html
+++
b/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-0-processors/src/main/resources/docs/org.apache.nifi.processors.kafka.pubsub.PublishKafkaRecord_2_0/additionalDetails.html
@@ -114,16 +114,65 @@
password="nifi-password";
};
</pre>
+ The JAAS configuration can be provided by either of below ways
+ <ol type="1">
+ <li>specify the java.security.auth.login.config system property in
+ NiFi's bootstrap.conf. This limits you to use only one user
credential across the cluster.</li>
+ <pre>
+
java.arg.16=-Djava.security.auth.login.config=/path/to/kafka_client_jaas.conf
+ </pre>
+ <li>add user attribute 'sasl.jaas.config' in the processor
configurations. This method allows one to have multiple consumers with
different user credentials or gives flexibility to consume from multiple kafka
clusters.</li>
+ <pre>
+ sasl.jaas.config :
org.apache.kafka.common.security.plain.PlainLoginModule required
+ username="nifi"
+ password="nifi-password";
+ </pre>
+ <b>NOTE:</b> The dynamic properties of this processor are not
secured and as a result the password entered when utilizing sasl.jaas.config
will be stored in the flow.xml.gz file in plain-text, and will be saved to NiFi
Registry if using versioned flows.
+ </ol>
</p>
<p>
<b>NOTE:</b> It is not recommended to use a SASL mechanism of
PLAIN with SASL_PLAINTEXT, as it would transmit
the username and password unencrypted.
</p>
<p>
+ <b>NOTE:</b> The Kerberos Service Name is not required for SASL
mechanism of PLAIN. However, processor warns saying this attribute has to be
filled with non empty string. You can choose to fill any random string, such as
"null".
+ </p>
+ <p>
<b>NOTE:</b> Using the PlainLoginModule will cause it be
registered in the JVM's static list of Providers, making
it visible to components in other NARs that may access the
providers. There is currently a known issue
where Kafka processors using the PlainLoginModule will cause HDFS
processors with Keberos to no longer work.
</p>
+
+ <h4>SASL_PLAINTEXT - SCRAM</h4>
+ <p>
+ If the SASL mechanism is SSL, then client must provide a JAAS
configuration to authenticate, but
+ the JAAS configuration must use Kafka's ScramLoginModule. Ensure
that you add user defined attribute 'sasl.mechanism' and assign 'SCRAM-SHA-256'
or 'SCRAM-SHA-512' based on kafka broker configurations. An example of the JAAS
config file would
+ be the following:
+ <pre>
+ KafkaClient {
+ org.apache.kafka.common.security.scram.ScramLoginModule
+ username="nifi"
+ password="nifi-password";
+ };
+ </pre>
+
+ The JAAS configuration can be provided by either of below ways
+ <ol type="1">
+ <li>specify the java.security.auth.login.config system property in
+ NiFi's bootstrap.conf. This limits you to use only one user
credential across the cluster.</li>
+ <pre>
+
java.arg.16=-Djava.security.auth.login.config=/path/to/kafka_client_jaas.conf
+ </pre>
+ <li>add user attribute 'sasl.jaas.config' in the processor
configurations. This method allows one to have multiple consumers with
different user credentials or gives flexibility to consume from multiple kafka
clusters.</li>
+ <pre>
+ sasl.jaas.config :
org.apache.kafka.common.security.scram.ScramLoginModule required
+ username="nifi"
+ password="nifi-password";
+ </pre>
+ <b>NOTE:</b> The dynamic properties of this processor are not
secured and as a result the password entered when utilizing sasl.jaas.config
will be stored in the flow.xml.gz file in plain-text, and will be saved to NiFi
Registry if using versioned flows.
+ </ol>
+ <b>NOTE:</b> The Kerberos Service Name is not required for SASL
mechanism of SCRAM-SHA-256 or SCRAM-SHA-512. However, processor warns saying
this attribute has to be filled with non empty string. You can choose to fill
any random string, such as "null".
+ </p>
<h3>SASL_SSL</h3>
<p>
This option uses SASL with an SSL/TLS transport layer to
authenticate to the broker. In order to use this
diff --git
a/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-0-processors/src/main/resources/docs/org.apache.nifi.processors.kafka.pubsub.PublishKafka_2_0/additionalDetails.html
b/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-0-processors/src/main/resources/docs/org.apache.nifi.processors.kafka.pubsub.PublishKafka_2_0/additionalDetails.html
index 1d26464..5618529 100644
---
a/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-0-processors/src/main/resources/docs/org.apache.nifi.processors.kafka.pubsub.PublishKafka_2_0/additionalDetails.html
+++
b/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-0-processors/src/main/resources/docs/org.apache.nifi.processors.kafka.pubsub.PublishKafka_2_0/additionalDetails.html
@@ -126,16 +126,65 @@
password="nifi-password";
};
</pre>
+ The JAAS configuration can be provided by either of below ways
+ <ol type="1">
+ <li>specify the java.security.auth.login.config system property in
+ NiFi's bootstrap.conf. This limits you to use only one user
credential across the cluster.</li>
+ <pre>
+
java.arg.16=-Djava.security.auth.login.config=/path/to/kafka_client_jaas.conf
+ </pre>
+ <li>add user attribute 'sasl.jaas.config' in the processor
configurations. This method allows one to have multiple consumers with
different user credentials or gives flexibility to consume from multiple kafka
clusters.</li>
+ <pre>
+ sasl.jaas.config :
org.apache.kafka.common.security.plain.PlainLoginModule required
+ username="nifi"
+ password="nifi-password";
+ </pre>
+ <b>NOTE:</b> The dynamic properties of this processor are not
secured and as a result the password entered when utilizing sasl.jaas.config
will be stored in the flow.xml.gz file in plain-text, and will be saved to NiFi
Registry if using versioned flows.
+ </ol>
</p>
<p>
<b>NOTE:</b> It is not recommended to use a SASL mechanism of
PLAIN with SASL_PLAINTEXT, as it would transmit
the username and password unencrypted.
</p>
<p>
+ <b>NOTE:</b> The Kerberos Service Name is not required for SASL
mechanism of PLAIN. However, processor warns saying this attribute has to be
filled with non empty string. You can choose to fill any random string, such as
"null".
+ </p>
+ <p>
<b>NOTE:</b> Using the PlainLoginModule will cause it be
registered in the JVM's static list of Providers, making
it visible to components in other NARs that may access the
providers. There is currently a known issue
where Kafka processors using the PlainLoginModule will cause HDFS
processors with Keberos to no longer work.
</p>
+
+ <h4>SASL_PLAINTEXT - SCRAM</h4>
+ <p>
+ If the SASL mechanism is SSL, then client must provide a JAAS
configuration to authenticate, but
+ the JAAS configuration must use Kafka's ScramLoginModule. Ensure
that you add user defined attribute 'sasl.mechanism' and assign 'SCRAM-SHA-256'
or 'SCRAM-SHA-512' based on kafka broker configurations. An example of the JAAS
config file would
+ be the following:
+ <pre>
+ KafkaClient {
+ org.apache.kafka.common.security.scram.ScramLoginModule
+ username="nifi"
+ password="nifi-password";
+ };
+ </pre>
+
+ The JAAS configuration can be provided by either of below ways
+ <ol type="1">
+ <li>specify the java.security.auth.login.config system property in
+ NiFi's bootstrap.conf. This limits you to use only one user
credential across the cluster.</li>
+ <pre>
+
java.arg.16=-Djava.security.auth.login.config=/path/to/kafka_client_jaas.conf
+ </pre>
+ <li>add user attribute 'sasl.jaas.config' in the processor
configurations. This method allows one to have multiple consumers with
different user credentials or gives flexibility to consume from multiple kafka
clusters.</li>
+ <pre>
+ sasl.jaas.config :
org.apache.kafka.common.security.scram.ScramLoginModule required
+ username="nifi"
+ password="nifi-password";
+ </pre>
+ <b>NOTE:</b> The dynamic properties of this processor are not
secured and as a result the password entered when utilizing sasl.jaas.config
will be stored in the flow.xml.gz file in plain-text, and will be saved to NiFi
Registry if using versioned flows.
+ </ol>
+ <b>NOTE:</b> The Kerberos Service Name is not required for SASL
mechanism of SCRAM-SHA-256 or SCRAM-SHA-512. However, processor warns saying
this attribute has to be filled with non empty string. You can choose to fill
any random string, such as "null".
+ </p>
<h3>SASL_SSL</h3>
<p>
This option uses SASL with an SSL/TLS transport layer to
authenticate to the broker. In order to use this
