Re: Datanode does not need hdfs.headless.keytab ?

[Di Li]

Great ! Thanks Rob!  I will try it out today and reach out if I hit issues.

Thank you for your help.

Di

On Wed, Apr 4, 2018 at 4:15 PM, Robert Levas  wrote:

> If you would like the HDFS keytab file installed on the same host as your
> component, you can add a reference to that Kerberos identity in your
> Kerberos.json file. Ideally this reference would be added to the
> "identities" section for the specific component.   The declaration would
> look something like this:
>
> {
>   "name": "custom_component_hdfs",
>   "reference": "/HDFS/NAMENODE/hdfs"
> }
>
> For example:
>
>   "components": [
> {
>   "name":  "MY_COMPONENT",
>   ...
>   "identities": [
> ...
> {
>   "name": "custom_component_hdfs ",
>   "reference": "/HDFS/NAMENODE/hdfs"
> }
> ...
>   ],
>   ...
> },
>
> I hope this helps.
>
> Rob
>
>
> On 4/4/18, 4:02 PM, "Di Li"  wrote:
>
> Hi Rob,
>
> Thanks for the explanation. I don't have issues with DN per se. My case
> falls into the "*since then some services need to create directories
> and
> change permissions on them as the HDFS root user upon installation *
> category
> that you mentioned. I paired my service with DN assuming
> hdfs.headless.keytab would be available.
>
> Is that possible for my service's kerberos.json to define a dependency
> on
> hdfs.headless.keytab ? This way, I can still cohost my component with
> DN
> (hard requirement ...) but still have the keytab available (no need to
> modify HDFS kerberos.json)
>
> Thanks.
>
> Di
>
> On Wed, Apr 4, 2018 at 3:17 PM, Robert Levas 
> wrote:
>
> > The DN does not need to authenticate as the "root" HDFS user to
> perform
> > administrative tasks.
> >
> > A while back, we started an initiative to reduce the exposure of the
> HDFS
> > "root" user due to security concerns.  In doing so, we tightened up
> where
> > we distribute the HDFS keytab file. However since then some services
> need
> > to create directories and change permissions on them as the HDFS
> root user
> > upon installation; and thus, the keytab file is being distributed
> more than
> > some security-conscious people would like.  Until we find a way to
> > centralize the creation of these HDFS resources, we need to deal
> with this.
> >
> > You should not normally need the HDFS keytab file on DN hosts... are
> you
> > having an issue?
> >
> > Rob
> >
> >
> > On 4/4/18, 2:15 PM, "Di Li"  wrote:
> >
> > Hi folks,
> >
> > I noticed hdfs.headless.keytab only exists on NameNode and HDFS
> client
> > node.
> >
> > Could someone please share some details on why DN does not need
> the
> > hdfs.headless.keytab ? I thought we need it in order for DN to
> work
> > against
> > NN.
> >
> > Any negative impacts if I always include hdfs.headless.keytab on
> the DN
> > nodes  (such as ensure HDFS client always cohost with DNs) ?
> >
> > Thank you.
> >
> > Di
> >
> >
> >
>
>
>

Re: Datanode does not need hdfs.headless.keytab ?

[Robert Levas]

If you would like the HDFS keytab file installed on the same host as your 
component, you can add a reference to that Kerberos identity in your 
Kerberos.json file. Ideally this reference would be added to the "identities" 
section for the specific component.   The declaration would look something like 
this:

{
  "name": "custom_component_hdfs",
  "reference": "/HDFS/NAMENODE/hdfs"
}

For example:

  "components": [
{
  "name":  "MY_COMPONENT",
  ...
  "identities": [
...
{
  "name": "custom_component_hdfs ",
  "reference": "/HDFS/NAMENODE/hdfs"
}
...
  ],
  ...
},

I hope this helps.

Rob


On 4/4/18, 4:02 PM, "Di Li"  wrote:

Hi Rob,

Thanks for the explanation. I don't have issues with DN per se. My case
falls into the "*since then some services need to create directories and
change permissions on them as the HDFS root user upon installation * 
category
that you mentioned. I paired my service with DN assuming
hdfs.headless.keytab would be available.

Is that possible for my service's kerberos.json to define a dependency on
hdfs.headless.keytab ? This way, I can still cohost my component with DN
(hard requirement ...) but still have the keytab available (no need to
modify HDFS kerberos.json)

Thanks.

Di

On Wed, Apr 4, 2018 at 3:17 PM, Robert Levas  wrote:

> The DN does not need to authenticate as the "root" HDFS user to perform
> administrative tasks.
>
> A while back, we started an initiative to reduce the exposure of the HDFS
> "root" user due to security concerns.  In doing so, we tightened up where
> we distribute the HDFS keytab file. However since then some services need
> to create directories and change permissions on them as the HDFS root user
> upon installation; and thus, the keytab file is being distributed more 
than
> some security-conscious people would like.  Until we find a way to
> centralize the creation of these HDFS resources, we need to deal with 
this.
>
> You should not normally need the HDFS keytab file on DN hosts... are you
> having an issue?
>
> Rob
>
>
> On 4/4/18, 2:15 PM, "Di Li"  wrote:
>
> Hi folks,
>
> I noticed hdfs.headless.keytab only exists on NameNode and HDFS client
> node.
>
> Could someone please share some details on why DN does not need the
> hdfs.headless.keytab ? I thought we need it in order for DN to work
> against
> NN.
>
> Any negative impacts if I always include hdfs.headless.keytab on the 
DN
> nodes  (such as ensure HDFS client always cohost with DNs) ?
>
> Thank you.
>
> Di
>
>
>

Re: Datanode does not need hdfs.headless.keytab ?

[Di Li]

Hi Rob,

Thanks for the explanation. I don't have issues with DN per se. My case
falls into the "*since then some services need to create directories and
change permissions on them as the HDFS root user upon installation * category
that you mentioned. I paired my service with DN assuming
hdfs.headless.keytab would be available.

Is that possible for my service's kerberos.json to define a dependency on
hdfs.headless.keytab ? This way, I can still cohost my component with DN
(hard requirement ...) but still have the keytab available (no need to
modify HDFS kerberos.json)

Thanks.

Di

On Wed, Apr 4, 2018 at 3:17 PM, Robert Levas  wrote:

> The DN does not need to authenticate as the "root" HDFS user to perform
> administrative tasks.
>
> A while back, we started an initiative to reduce the exposure of the HDFS
> "root" user due to security concerns.  In doing so, we tightened up where
> we distribute the HDFS keytab file. However since then some services need
> to create directories and change permissions on them as the HDFS root user
> upon installation; and thus, the keytab file is being distributed more than
> some security-conscious people would like.  Until we find a way to
> centralize the creation of these HDFS resources, we need to deal with this.
>
> You should not normally need the HDFS keytab file on DN hosts... are you
> having an issue?
>
> Rob
>
>
> On 4/4/18, 2:15 PM, "Di Li"  wrote:
>
> Hi folks,
>
> I noticed hdfs.headless.keytab only exists on NameNode and HDFS client
> node.
>
> Could someone please share some details on why DN does not need the
> hdfs.headless.keytab ? I thought we need it in order for DN to work
> against
> NN.
>
> Any negative impacts if I always include hdfs.headless.keytab on the DN
> nodes  (such as ensure HDFS client always cohost with DNs) ?
>
> Thank you.
>
> Di
>
>
>

Re: Datanode does not need hdfs.headless.keytab ?

[Robert Levas]

The DN does not need to authenticate as the "root" HDFS user to perform 
administrative tasks.  

A while back, we started an initiative to reduce the exposure of the HDFS 
"root" user due to security concerns.  In doing so, we tightened up where we 
distribute the HDFS keytab file. However since then some services need to 
create directories and change permissions on them as the HDFS root user upon 
installation; and thus, the keytab file is being distributed more than some 
security-conscious people would like.  Until we find a way to centralize the 
creation of these HDFS resources, we need to deal with this.  

You should not normally need the HDFS keytab file on DN hosts... are you having 
an issue? 

Rob


On 4/4/18, 2:15 PM, "Di Li"  wrote:

Hi folks,

I noticed hdfs.headless.keytab only exists on NameNode and HDFS client
node.

Could someone please share some details on why DN does not need the
hdfs.headless.keytab ? I thought we need it in order for DN to work against
NN.

Any negative impacts if I always include hdfs.headless.keytab on the DN
nodes  (such as ensure HDFS client always cohost with DNs) ?

Thank you.

Di