[
https://issues.apache.org/jira/browse/ATLAS-4361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17381652#comment-17381652
]
ighack edited comment on ATLAS-4361 at 7/19/21, 9:31 AM:
---------------------------------------------------------
I modify KafkaNotification.java
add info
{code:java}
if (ret == null) {
LOG.info("properties is" + properties.toString());
ret = new KafkaProducer(properties);
producers.put(notificationType, ret);
}
{code}
I get
properties is:\{sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule
required keyTab="/hadoop/app/jztwk.keytab" storeKey=true useKeyTab=true
serviceName=kafka principal="jztwk/[email protected]" ;,
key.deserializer=org.apache.kafka.common.serialization.StringDeserializer,
value.deserializer=org.apache.kafka.common.serialization.StringDeserializer,
key.serializer=org.apache.kafka.common.serialization.StringSerializer,
zookeeper.session.timeout.ms=400, poll.timeout.ms=1000,
enable.auto.commit=false, security.protocol=SASL_PLAINTEXT, group.id=atlas,
auto.offset.reset=earliest, offsets.topic.replication.factor=1,
zookeeper.sync.time.ms=20, max.poll.records=1,
bootstrap.servers=bigdser1:9092,bigdser2:9092,bigdser3:9092,bigdser4:9092,bigdser5:9092,
session.timeout.ms=30000, sasl.kerberos.service.name=kafka,
value.serializer=org.apache.kafka.common.serialization.StringSerializer,
auto.commit.interval.ms=1000, zookeeper.connection.timeout.ms=200,
data=${sys:atlas.home}/data/kafka,
zookeeper.connect=bigdser3:2181,bigdser2:2181,bigdser5:2181}
and I am sure /hadoop/app/jztwk.keytab [and
jztwk/[email protected]|mailto:and%C2%A0jztwk/[email protected]] is correct
CDH-6.3.2
atlas
release-2.1.0-rc3
was (Author: ighack):
[Atlas Notifier 0]: ProducerConfig values:
acks = 1
batch.size = 16384
bootstrap.servers = [bigdser1:9092, bigdser2:9092, bigdser3:9092,
bigdser4:9092, bigdser5:9092]
buffer.memory = 33554432
client.dns.lookup = default
client.id =
compression.type = none
connections.max.idle.ms = 540000
delivery.timeout.ms = 120000
enable.idempotence = false
interceptor.classes = []
key.serializer = class org.apache.kafka.common.serialization.StringSerializer
linger.ms = 0
max.block.ms = 60000
max.in.flight.requests.per.connection = 5
max.request.size = 1048576
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partitioner.class = class
org.apache.kafka.clients.producer.internals.DefaultPartitioner
receive.buffer.bytes = 32768
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retries = 2147483647
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = [hidden]
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = kafka
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = SASL_PLAINTEXT
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
transaction.timeout.ms = 60000
transactional.id = null
value.serializer = class org.apache.kafka.common.serialization.StringSerializer
> Hive Hook can't send notification
> ---------------------------------
>
> Key: ATLAS-4361
> URL: https://issues.apache.org/jira/browse/ATLAS-4361
> Project: Atlas
> Issue Type: Bug
> Components: hive-integration
> Affects Versions: 2.1.0
> Reporter: ighack
> Priority: Blocker
> Attachments: 20210716165232274.png
>
>
> when I create a view
> I get a error
>
> Caused by: javax.security.auth.login.LoginException: Could not login: the
> client is being asked for a password, but the Kafka client code does not
> currently support obtaining a password from the user. not available to garner
> authentication information from the user
>
> but I make sure keytab and principal is correct
> #Enable for Kerberized Kafka clusters
> atlas.notification.kafka.service.principal=jztwk/[email protected]
> atlas.notification.kafka.keytab.location=/hadoop/app/jztwk.keytab
> atlas.kafka.security.protocol=SASL_PLAINTEXT
> atlas.kafka.sasl.kerberos.service.name=kafka
> ##############JAAS Configuration ########
> atlas.jaas.KafkaClient.loginModuleName=com.sun.security.auth.module.Krb5LoginModule
> atlas.jaas.KafkaClient.loginModuleControlFlag=required
> atlas.jaas.KafkaClient.option.useKeyTab=true
> atlas.jaas.KafkaClient.option.storeKey=true
> atlas.jaas.KafkaClient.option.serviceName=kafka
> atlas.jaas.KafkaClient.option.keyTab=/hadoop/app/jztwk.keytab
>
> [atlas.jaas.KafkaClient.option.principal=jztwk/[email protected]|mailto:atlas.jaas.KafkaClient.option.principal=jztwk/[email protected]]
>
> and I modify
> [KafkaNotification.java|https://sourcegraph.com/github.com/apache/atlas/-/blob/notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java]
> add a debug
> LOG.debug("sasl.jaas.config is " + newJaasProperty);
>
> I see
> sasl.jaas.config is *com.sun.security.auth.module.Krb5LoginModule required
> keyTab="/hadoop/app/jztwk.keytab" storeKey=true useKeyTab=true
> principal="jztwk/[email protected]" ;*
> in log
> I use this config to my application.
> that is OK.
>
> Properties props = new Properties();
> props.put("bootstrap.servers",
> "bigdser1:9092,bigdser2:9092,bigdser3:9092,bigdser4:9092,bigdser5:9092");
> props.put("sasl.jaas.config","com.sun.security.auth.module.Krb5LoginModule
> required keyTab=\"/hadoop/jztwk.keytab\" storeKey=true useKeyTab=true
> principal=\"jztwk/[email protected]\" ;");
> props.put("security.protocol", "SASL_PLAINTEXT");
> props.put("sasl.kerberos.service.name", "kafka");
> props.put("key.serializer",
> "org.apache.kafka.common.serialization.StringSerializer");
> props.put("value.serializer",
> "org.apache.kafka.common.serialization.StringSerializer");
> KafkaProducer<String,String> producer = new KafkaProducer<>(props);
>
> and I can run import-hive.sh
> so, I don't understand why
>
> all node I have run *kinit -kt /hadoop/app/jztwk.keytab
> [jztwk/[email protected]|mailto:jztwk/[email protected]]*
> -rwxr-xr-x 1 jzyc jzyc 146 Jul 14 08:46 /hadoop/app/jztwk.keytab
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
