Re: Enable Kerberos on Ignite Kafka streamer / support for new consumer config

[jackson.dickson]

Thanks Andrew.

The new consumer config enable Kafka clients to connect directly to the
Kafka brokers. kafka-clients 0.10.x support both style. I believe you would
need another ConsumerConfig type like ZKConfig.

-Jackson



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Enable Kerberos on Ignite Kafka streamer / support for new consumer config

[Andrey Mashenkov]

Hi Jackson,

Got it. I've created a ticket for this [1].
I'm not too close familiar with Kafka. Please, let me know if I missed smth.

Seems, the fix is trivial, we've just create a KafkaStreamer directly
instead of using zookeeper kafka connector.

[1] https://issues.apache.org/jira/browse/IGNITE-7752

On Mon, Feb 19, 2018 at 2:04 PM, jackson.dickson 
wrote:

> Hi Andrew,
>
> Yes, "bootstrap.server" option is not working for me.
>
> -Jackson
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov

Re: Enable Kerberos on Ignite Kafka streamer / support for new consumer config

[jackson.dickson]

Hi Andrew,

Yes, "bootstrap.server" option is not working for me.

-Jackson



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Enable Kerberos on Ignite Kafka streamer / support for new consumer config

[jackson.dickson]

Hi Andrew,

Yes, "bootstrap.server" option is not working for you?

-Jackson



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Enable Kerberos on Ignite Kafka streamer / support for new consumer config

[Andrey Mashenkov]

Hi Jackson,

I can't figure out how to switch to new style config.
Seem, "bootstrap.servers" option should be used instead of
"zookeeper.connect".
Do you mean using "bootstrap.server" option is not working for you?



On Wed, Feb 7, 2018 at 1:18 PM, jackson.dickson 
wrote:

> Thanks Andrew for your response.
>
> As per this documentation (
> https://docs.confluent.io/3.0.0/kafka/sasl.html#sasl-
> configuration-for-kafka-clients
>  configuration-for-kafka-clients>
> ), SASL auth is supported only for new Kafka consumers. The new kafka
> consumers connect to the Kafka brokers directly instead of zookeeper
>
>
> Below two security config are required to be specified in the consumers
> configs which are only supported in the new Kafka consumer config.
> security.protocol
> sasl.kerberos.service.name
>
> Refer -  https://kafka.apache.org/0100/documentation.html#
> oldconsumerconfigs
> 
>
> The ignite kafka streamer uses old consumer config
> (kafka.consumer.ConsumerConfig). I get below errors in the log file if I
> specify these new consumer configs.
>
> 2018-02-07 05:11:42.432|myapp|ip-x-x-x-x.myserver|[main]|INFO
> |||kafka.utils.VerifiableProperties:info|68|Verifying properties
> 2018-02-07 05:11:42.454|myapp|ip-x-x-x-x.myserver|[main]|WARN
> |||kafka.utils.VerifiableProperties:warn|83|Property auto.reset.offset
> is not valid
> 2018-02-07 05:11:42.455|myapp|ip-x-x-x-x.myserver|[main]|INFO
> |||kafka.utils.VerifiableProperties:info|68|Property group.id is
> overridden to ignio-kafka-streamer
> 2018-02-07 05:11:42.455|myapp|ip-x-x-x-x.myserver|[main]|WARN
> |||kafka.utils.VerifiableProperties:warn|83|Property
> sasl.kerberos.service.name is not valid
> 2018-02-07 05:11:42.455|myapp|ip-x-x-x-x.myserver|[main]|WARN
> |||kafka.utils.VerifiableProperties:warn|83|Property security.protocol
> is not valid
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov

Re: Enable Kerberos on Ignite Kafka streamer / support for new consumer config

[jackson.dickson]

Thanks Andrew for your response.

As per this documentation (
https://docs.confluent.io/3.0.0/kafka/sasl.html#sasl-configuration-for-kafka-clients

 
), SASL auth is supported only for new Kafka consumers. The new kafka
consumers connect to the Kafka brokers directly instead of zookeeper


Below two security config are required to be specified in the consumers
configs which are only supported in the new Kafka consumer config.
security.protocol
sasl.kerberos.service.name

Refer -  https://kafka.apache.org/0100/documentation.html#oldconsumerconfigs
  

The ignite kafka streamer uses old consumer config
(kafka.consumer.ConsumerConfig). I get below errors in the log file if I
specify these new consumer configs.

2018-02-07 05:11:42.432|myapp|ip-x-x-x-x.myserver|[main]|INFO
|||kafka.utils.VerifiableProperties:info|68|Verifying properties
2018-02-07 05:11:42.454|myapp|ip-x-x-x-x.myserver|[main]|WARN
|||kafka.utils.VerifiableProperties:warn|83|Property auto.reset.offset
is not valid
2018-02-07 05:11:42.455|myapp|ip-x-x-x-x.myserver|[main]|INFO
|||kafka.utils.VerifiableProperties:info|68|Property group.id is
overridden to ignio-kafka-streamer
2018-02-07 05:11:42.455|myapp|ip-x-x-x-x.myserver|[main]|WARN
|||kafka.utils.VerifiableProperties:warn|83|Property
sasl.kerberos.service.name is not valid
2018-02-07 05:11:42.455|myapp|ip-x-x-x-x.myserver|[main]|WARN
|||kafka.utils.VerifiableProperties:warn|83|Property security.protocol
is not valid



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Enable Kerberos on Ignite Kafka streamer / support for new consumer config

[Andrey Mashenkov]

Hi Jackson,

Ignite supports kafka-0.10.0.1 version.
Kafka official docs has support some kerberos options [1].
So, it should be possible to enable Kerberos with certain options in
ConsumerConfig.


[1] https://kafka.apache.org/0100/documentation.html#brokerconfigs

On Wed, Feb 7, 2018 at 7:49 AM, jackson.dickson 
wrote:

> Hi,
>
> We are trying to enable Kerberos in our Ignite-Kafka env. Kerberos is
> supported in Kafka consumers using the newer consumer configs. As per Kafka
> documentation, Kerberos is only supported on the newer KAfka consumers.
>
> It seems the Ignite KafkaStreamer only supports using the old consumer
> config.
>
> Is there any way to setup Kerberos auth on KafkaStreamer?
>
> Thanks,
> Jackson
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov

Enable Kerberos on Ignite Kafka streamer / support for new consumer config

[jackson.dickson]

Hi,

We are trying to enable Kerberos in our Ignite-Kafka env. Kerberos is
supported in Kafka consumers using the newer consumer configs. As per Kafka
documentation, Kerberos is only supported on the newer KAfka consumers.

It seems the Ignite KafkaStreamer only supports using the old consumer
config.

Is there any way to setup Kerberos auth on KafkaStreamer?

Thanks,
Jackson



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/