Re: Issue in Flink/Zookeeper authentication via Kerberos

2018-04-15 Thread Eron Wright 
I believe that the solution here is to ensure that the znodes created by
Flink have an ACL that allows access only to the original creator.   For
example, if a given Flink job has a Kerberos identity of "us...@example.com",
it should set the znode ACL appropriately to disallow access to any client
that doesn't successfully authenticate as that user.  This may be
accomplished with the following Flink configuration setting:

high-availability.zookeeper.client.acl: creator

Some code links:
-
https://github.com/apache/flink/blob/release-1.4.2/flink-core/src/main/java/org/apache/flink/configuration/HighAvailabilityOptions.java#L171
-
https://github.com/apache/flink/blob/release-1.4.2/flink-runtime/src/main/java/org/apache/flink/runtime/util/ZooKeeperUtils.java#L93

Hope this helps!
Eron

On Sun, Apr 15, 2018 at 2:16 AM, Sahu, Sarthak 1. (Nokia - IN/Bangalore) <
sarthak.1.s...@nokia.com> wrote:

> Glad to get the reply. With wrong Kerberos information I am expecting an
> ‘access denied’.
>
>
>
> As per flink log, it clear states that authentication failed due to
> Kerberos wrong information and trying to connect with zookeeper with
> unauthorised mode if zookeeper allows.
>
> And then it connected successfully!
>
>
>
> Do I missing any configuration in flink/zookeeper side.
>
> Expecting you suggestion here.
>
>
>
> Regards
>
> Sarthak Sahu
>
>
>
> *From:* Eron Wright [mailto:eronwri...@gmail.com]
> *Sent:* Tuesday, April 3, 2018 3:07 AM
> *To:* Sahu, Sarthak 1. (Nokia - IN/Bangalore) <sarthak.1.s...@nokia.com>
> *Cc:* suez1...@gmail.com; Timo Walther <twal...@apache.org>
>
> *Subject:* Re: Issue in Flink/Zookeeper authentication via Kerberos
>
>
>
> Hello, I'm happy to help.  Could you elaborate on the issue that you see?
> Are you saying that you expect to get 'access denied' but Zookeeper is
> allowing the connection anyway?
>
>
>
> My first thought is, maybe ZK allows unauthenticated connections but
> relies on the authorization layer to deny access to nodes based on the
> ACL.   FLink has a configuration setting to set the 'owner' of the znode.
>
>
>
> -Eron
>
>
>
> On Mon, Apr 2, 2018 at 1:50 AM, Sahu, Sarthak 1. (Nokia - IN/Bangalore) <
> sarthak.1.s...@nokia.com> wrote:
>
> Hi Eron/Shuyi
>
>
>
> Could you please help me on this below issue.
>
>
>
> Regards
>
> Sarthak Sahu
>
>
>
> *From:* Timo Walther [mailto:twal...@apache.org]
> *Sent:* Monday, March 26, 2018 3:05 PM
> *To:* user@flink.apache.org
> *Cc:* eronwri...@gmail.com; suez1...@gmail.com
> *Subject:* Re: Issue in Flink/Zookeeper authentication via Kerberos
>
>
>
> Hi Sarthak,
>
> I'm not a Kerberos expert but maybe Eron or Shuyi are more familiar with
> the details?
>
> Would be great if somebody could help.
>
> Thanks,
> Timo
>
> Am 22.03.18 um 10:16 schrieb Sahu, Sarthak 1. (Nokia - IN/Bangalore):
>
> Hi Folks,
>
>
>
>   *Environment Setup:*
>
>1. I have configured KDC 5 server.
>2. Configured Kerberos in zookeeper-3.4.10 wherein I can able to
>connect ZooKeeper Server/Client via Kerberos authentication.
>3. Now flink-1.4.0 has configured for Kerberos authentication as per
>below instruction.
>
> ·   https://ci.apache.org/projects/flink/flink-docs-
> release-1.4/ops/config.html#kerberos-based-security
>
> ·   https://ci.apache.org/projects/flink/flink-docs-
> release-1.4/ops/config.html#kerberos-based-security-1
>
>   *Success Scenario:*
>
>1. All Kerberos configuration parameter is correct and flink/zookeeper
>able to connect trough TGT.
>
>  *Problem:*
>
>1. Even if wrong Kerberos credentials given, flink able to connect
>ZooKeeper.
>
>
>
> Please find the taskmanager/jobmanger logs and flink config file for both
> scenario attached.
>
>
>
> Hoping for quick resolution.
>
>
>
> Regards
>
> Sarthak Sahu
>
>
>
>
>
>
>

Re: Issue in Flink/Zookeeper authentication via Kerberos

2018-04-02 Thread Shuyi Chen 
Hi Sarthak,

Happy to help. Could you please share the jobmanager/taskmanager log and
flink conf again?

Also, Flink 1.4.0 has a regression on kerberos security (keytab path in
TaskManager is set incorrectly) , which is fixed on 1.4.1. (see
https://issues.apache.org/jira/browse/FLINK-8275)

Shuyi

On Mon, Apr 2, 2018 at 3:44 PM, Shuyi Chen  wrote:

> Hi Sarthak,
>
> Happy to help. Could you please share the jobmanager/taskmanager log and
> flink conf again?
>
> Also, Flink 1.4.0 has a regression on kerberos security (keytab path in
> TaskManager is set incorrectly) , which is fixed on 1.4.1. (see
> https://issues.apache.org/jira/browse/FLINK-8275)
>
> Shuyi
>
> On Mon, Mar 26, 2018 at 2:35 AM, Timo Walther  wrote:
>
>> Hi Sarthak,
>>
>> I'm not a Kerberos expert but maybe Eron or Shuyi are more familiar with
>> the details?
>>
>> Would be great if somebody could help.
>>
>> Thanks,
>> Timo
>>
>> Am 22.03.18 um 10:16 schrieb Sahu, Sarthak 1. (Nokia - IN/Bangalore):
>>
>> Hi Folks,
>>
>>
>>
>>   *Environment Setup:*
>>
>>1. I have configured KDC 5 server.
>>2. Configured Kerberos in zookeeper-3.4.10 wherein I can able to
>>connect ZooKeeper Server/Client via Kerberos authentication.
>>3. Now flink-1.4.0 has configured for Kerberos authentication as per
>>below instruction.
>>
>>
>>- https://ci.apache.org/projects/flink/flink-docs-release-1.4/
>>ops/config.html#kerberos-based-security
>>
>> 
>>- https://ci.apache.org/projects/flink/flink-docs-release-1.4/
>>ops/config.html#kerberos-based-security-1
>>
>> 
>>
>>   *Success Scenario:*
>>
>>1. All Kerberos configuration parameter is correct and
>>flink/zookeeper able to connect trough TGT.
>>
>>  *Problem:*
>>
>>1. Even if wrong Kerberos credentials given, flink able to connect
>>ZooKeeper.
>>
>>
>>
>> Please find the taskmanager/jobmanger logs and flink config file for both
>> scenario attached.
>>
>>
>>
>> Hoping for quick resolution.
>>
>>
>>
>> Regards
>>
>> Sarthak Sahu
>>
>>
>>
>>
>>
>
>
> --
> "So you have to trust that the dots will somehow connect in your future."
>



-- 
"So you have to trust that the dots will somehow connect in your future."

Re: Issue in Flink/Zookeeper authentication via Kerberos

2018-03-26 Thread Timo Walther 

Hi Sarthak,

I'm not a Kerberos expert but maybe Eron or Shuyi are more familiar with 
the details?


Would be great if somebody could help.

Thanks,
Timo

Am 22.03.18 um 10:16 schrieb Sahu, Sarthak 1. (Nokia - IN/Bangalore):


Hi Folks,

*_Environment Setup:_*

 1. I have configured KDC 5 server.
 2. Configured Kerberos in zookeeper-3.4.10 wherein I can able to
connect ZooKeeper Server/Client via Kerberos authentication.
 3. Now flink-1.4.0 has configured for Kerberos authentication as per
below instruction.

  * 
https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/config.html#kerberos-based-security
  * 
https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/config.html#kerberos-based-security-1

*_Success Scenario:_*

 1. All Kerberos configuration parameter is correct and
flink/zookeeper able to connect trough TGT.

*_Problem:_*

 1. Even if wrong Kerberos credentials given, flink able to connect
ZooKeeper.*__*

*__*

Please find the taskmanager/jobmanger logs and flink config file for 
both scenario attached.


Hoping for quick resolution.

Regards

Sarthak Sahu