[jira] [Updated] (HADOOP-14627) Support MSI and DeviceCode token provider in ADLS
[ https://issues.apache.org/jira/browse/HADOOP-14627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-14627: Resolution: Fixed Fix Version/s: 2.8.3 3.0.0-beta1 2.9.0 Status: Resolved (was: Patch Available) Committed to trunk, branch-2, and branch-2.8. Thanks [~ASikaria] for the contribution! > Support MSI and DeviceCode token provider in ADLS > - > > Key: HADOOP-14627 > URL: https://issues.apache.org/jira/browse/HADOOP-14627 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/adl > Environment: MSI Change applies only to Hadoop running in an Azure VM >Reporter: Atul Sikaria >Assignee: Atul Sikaria > Fix For: 2.9.0, 3.0.0-beta1, 2.8.3 > > Attachments: HADOOP-14627-001.patch, HADOOP-14627.002.patch, > HADOOP-14627.003.patch, HADOOP-14627.004.patch > > > This change is to upgrade the Hadoop ADLS connector to enable new auth > features exposed by the ADLS Java SDK. > Specifically: > MSI Tokens: MSI (Managed Service Identity) is a way to provide an identity to > an Azure Service. In the case of VMs, they can be used to give an identity to > a VM deployment. This simplifies managing Service Principals, since the creds > don’t have to be managed in core-site files anymore. The way this works is > that during VM deployment, the ARM (Azure Resource Manager) template needs to > be modified to enable MSI. Once deployed, the MSI extension runs a service on > the VM that exposes a token endpoint to http://localhost at a port specified > in the template. The SDK has a new TokenProvider to fetch the token from this > local endpoint. This change would expose that TokenProvider as an auth option. > DeviceCode auth: This enables a token to be obtained from an interactive > login. The user is given a URL and a token to use on the login screen. User > can use the token to login from any device. Once the login is done, the token > that is obtained is in the name of the user who logged in. Note that because > of the interactive login involved, this is not very suitable for job > scenarios, but can work for ad-hoc scenarios like running “hdfs dfs” commands. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14627) Support MSI and DeviceCode token provider in ADLS
[ https://issues.apache.org/jira/browse/HADOOP-14627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-14627: Attachment: HADOOP-14627.004.patch Patch 004 * Trivial fix in index.md to take care of HADOOP-14438 > Support MSI and DeviceCode token provider in ADLS > - > > Key: HADOOP-14627 > URL: https://issues.apache.org/jira/browse/HADOOP-14627 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/adl > Environment: MSI Change applies only to Hadoop running in an Azure VM >Reporter: Atul Sikaria >Assignee: Atul Sikaria > Attachments: HADOOP-14627-001.patch, HADOOP-14627.002.patch, > HADOOP-14627.003.patch, HADOOP-14627.004.patch > > > This change is to upgrade the Hadoop ADLS connector to enable new auth > features exposed by the ADLS Java SDK. > Specifically: > MSI Tokens: MSI (Managed Service Identity) is a way to provide an identity to > an Azure Service. In the case of VMs, they can be used to give an identity to > a VM deployment. This simplifies managing Service Principals, since the creds > don’t have to be managed in core-site files anymore. The way this works is > that during VM deployment, the ARM (Azure Resource Manager) template needs to > be modified to enable MSI. Once deployed, the MSI extension runs a service on > the VM that exposes a token endpoint to http://localhost at a port specified > in the template. The SDK has a new TokenProvider to fetch the token from this > local endpoint. This change would expose that TokenProvider as an auth option. > DeviceCode auth: This enables a token to be obtained from an interactive > login. The user is given a URL and a token to use on the login screen. User > can use the token to login from any device. Once the login is done, the token > that is obtained is in the name of the user who logged in. Note that because > of the interactive login involved, this is not very suitable for job > scenarios, but can work for ad-hoc scenarios like running “hdfs dfs” commands. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14627) Support MSI and DeviceCode token provider in ADLS
[ https://issues.apache.org/jira/browse/HADOOP-14627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-14627: Attachment: HADOOP-14627.003.patch Patch 003 * Fix TestAzureADTokenProvider.testMSITokenProvider failure * My last review comments * Formatting in core-default.xml and index.md Testing Done * Passed live unit tests > Support MSI and DeviceCode token provider in ADLS > - > > Key: HADOOP-14627 > URL: https://issues.apache.org/jira/browse/HADOOP-14627 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/adl > Environment: MSI Change applies only to Hadoop running in an Azure VM >Reporter: Atul Sikaria >Assignee: Atul Sikaria > Attachments: HADOOP-14627-001.patch, HADOOP-14627.002.patch, > HADOOP-14627.003.patch > > > This change is to upgrade the Hadoop ADLS connector to enable new auth > features exposed by the ADLS Java SDK. > Specifically: > MSI Tokens: MSI (Managed Service Identity) is a way to provide an identity to > an Azure Service. In the case of VMs, they can be used to give an identity to > a VM deployment. This simplifies managing Service Principals, since the creds > don’t have to be managed in core-site files anymore. The way this works is > that during VM deployment, the ARM (Azure Resource Manager) template needs to > be modified to enable MSI. Once deployed, the MSI extension runs a service on > the VM that exposes a token endpoint to http://localhost at a port specified > in the template. The SDK has a new TokenProvider to fetch the token from this > local endpoint. This change would expose that TokenProvider as an auth option. > DeviceCode auth: This enables a token to be obtained from an interactive > login. The user is given a URL and a token to use on the login screen. User > can use the token to login from any device. Once the login is done, the token > that is obtained is in the name of the user who logged in. Note that because > of the interactive login involved, this is not very suitable for job > scenarios, but can work for ad-hoc scenarios like running “hdfs dfs” commands. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14627) Support MSI and DeviceCode token provider in ADLS
[ https://issues.apache.org/jira/browse/HADOOP-14627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-14627: Summary: Support MSI and DeviceCode token provider in ADLS (was: Support MSI and DeviceCode token provider) > Support MSI and DeviceCode token provider in ADLS > - > > Key: HADOOP-14627 > URL: https://issues.apache.org/jira/browse/HADOOP-14627 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/adl > Environment: MSI Change applies only to Hadoop running in an Azure VM >Reporter: Atul Sikaria >Assignee: Atul Sikaria > Attachments: HADOOP-14627-001.patch, HADOOP-14627.002.patch > > > This change is to upgrade the Hadoop ADLS connector to enable new auth > features exposed by the ADLS Java SDK. > Specifically: > MSI Tokens: MSI (Managed Service Identity) is a way to provide an identity to > an Azure Service. In the case of VMs, they can be used to give an identity to > a VM deployment. This simplifies managing Service Principals, since the creds > don’t have to be managed in core-site files anymore. The way this works is > that during VM deployment, the ARM (Azure Resource Manager) template needs to > be modified to enable MSI. Once deployed, the MSI extension runs a service on > the VM that exposes a token endpoint to http://localhost at a port specified > in the template. The SDK has a new TokenProvider to fetch the token from this > local endpoint. This change would expose that TokenProvider as an auth option. > DeviceCode auth: This enables a token to be obtained from an interactive > login. The user is given a URL and a token to use on the login screen. User > can use the token to login from any device. Once the login is done, the token > that is obtained is in the name of the user who logged in. Note that because > of the interactive login involved, this is not very suitable for job > scenarios, but can work for ad-hoc scenarios like running “hdfs dfs” commands. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
