Thanks again. So this might be very telling of the underlying problem:
I did what you suggested:
1) I disabled (actually deleted) the first rule; then
2) I changed the load balancer's second (which is now its only) rule to accept
TCP:9093 and to translate that to TCP:9093, making the conneciton PLAINTEXT all
the way through to Kafka; then
3) I tried connecting a Scala consumer to the load balancer URL
(mybalancer01.example.com) and I'm getting that ClosedChannelException
For now there is only one Kafka broker sitting behind the load balancer. It's
server.properties look like:
listeners=PLAINTEXT://:9093,SASL_PLAINTEXT://:9092
advertised.listeners=PLAINTEXT://mybalancer01.example.com:9093,SASL_PLAINTEXT://mykafka01.example.com:9092
advertised.host.name=mykafka01.example.com
security.inter.broker.protocol=SASL_PLAINTEXT
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
broker.id=1
num.partitions=4
zookeeper.connect=zkA:2181,zkB:2181,zkC:2181
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
log.dirs=/tmp/kafka-logs
num.recovery.threads.per.data.dir=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connection.timeout.ms=6000
offset.metadata.max.bytes=4096
Above, 'zkA', 'zkB' and 'zkC' are defined inside `/etc/hosts` and are valid
server names.
And then inside the kafka-run-class.sh script, instead of the default:
if [ -z "$KAFKA_OPTS" ]; then
KAFKA_OPTS=""
fi
I have:
if [ -z "$KAFKA_OPTS" ]; then
KAFKA_OPTS="-Djava.security.auth.login.config=/opt/kafka/config/jaas.conf"
fi
I also added the /opt/kafka/config/jaas.conf file like you suggested, and only
changed the names of users and passwords:
KafkaServer {
org.apache.kafka.common.security.plain.PlainLoginModule required
username="someuser"
user_kafka="somePassword"
password="kafka-password";
};
The fact that I can no longer even consume from a topic over PLAINTEXT (which
is a regression of where I was before we started trying to add SSL) tells me
there is something wrong in either server.properties or jaas.conf. I've checked
the Kafka broker logs (server.log) each time I try connecting and this is the
only line that gets printed:
[2016-11-21 15:18:14,859] INFO [Group Metadata Manager on Broker 2]: Removed 0
expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)
Not sure if that means anything. Any idea where I might be going wrong? Thanks
again!
________________________________
From: Rajini Sivaram <rajinisiva...@googlemail.com>
Sent: Monday, November 21, 2016 11:03:14 AM
To: users@kafka.apache.org
Subject: Re: Can Kafka/SSL be terminated at a load balancer?
Rule #1 and Rule #2 cannot co-exist. You are basically configuring your LB
to point to a Kafka broker and you are pointing each Kafka broker to point
to a LB. So you need a pair of ports with a security protocol for the
connection to work. With two rules, Kafka picks up the wrong LB port for
one of the security protocols.
If you want to try without SSL first, the simplest way to try it out would
be to disable Rule #1 and change Rule #2 to use port 9093 instead of 9095.
Then you should be able to connect using PLAINTEXT (the test that is
currently not working).
I think you have the configuration:
advertised.listeners=PLAINTEXT://mybalancer01.example.com:9093
,SASL_PLAINTEXT://mykafka01.example.com:9092
And you have a client connecting with PLAINTEXT on mybalancer01:*9095*. The
first connection would work, but subsequent connections would use the
address provided by Kafka from advertised.listeners. The client will start
connecting with PLAINTEXT on mybalancer01:*9093*, which is expecting SSL.
If you disable Rule #1 and change Rule #2 to use port 9093, you should be
able to test PLAINTEXT without changing Kafka config.
On Mon, Nov 21, 2016 at 3:32 PM, Zac Harvey <zac.har...@welltok.com> wrote:
> In the last email I should have mentioned: don't pay too much attention to
> the code snippet, and after reviewing it, I can see it actually incomplete
> (I forgot to include the section where I configure the topics and broker
> configs to talk to Kafka!).
>
>
> What I'm really concerned about is that before we added all these SSL
> configs, I had plaintext (plaintext:9092 in/out of the load balancer
> to/from Kafka) working fine. Now my consumer code can't even connect to the
> load balancer/Kafka.
>
>
> So I guess what I was really asking was: does that exception
> (ClosedChannelException) indicate bad configs on the Kafka broker?
>
> ________________________________
> From: Zac Harvey <zac.har...@welltok.com>
> Sent: Thursday, November 17, 2016 4:44:06 PM
> To: users@kafka.apache.org
> Subject: Can Kafka/SSL be terminated at a load balancer?
>
> We have two Kafka nodes and for reasons outside of this question, would
> like to set up a load balancer to terminate SSL with producers (clients).
> The SSL cert hosted by the load balancer will be signed by trusted/root CA
> that clients should natively trust.
>
>
> Is this possible to do, or does Kafka somehow require SSL to be setup
> directly on the Kafka servers themselves?
>
>
> Thanks!
>
--
Regards,
Rajini
