[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=319898=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319898
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 27/Sep/19 23:53
Start Date: 27/Sep/19 23:53
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1511:
HDDS-2162. Make Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r329284058
##
File path:
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerConfiguration.java
##
@@ -119,10 +119,13 @@ public void testDefaultPortIfNotSpecified() throws
Exception {
String omNode1Id = "omNode1";
String omNode2Id = "omNode2";
String omNodesKeyValue = omNode1Id + "," + omNode2Id;
-conf.set(OMConfigKeys.OZONE_OM_NODES_KEY, omNodesKeyValue);
+String serviceID = "service1";
+conf.set(OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY, serviceID);
+conf.set(OMConfigKeys.OZONE_OM_NODES_KEY + "." + serviceID,
+omNodesKeyValue);
-String omNode1RpcAddrKey = getOMAddrKeyWithSuffix(null, omNode1Id);
-String omNode2RpcAddrKey = getOMAddrKeyWithSuffix(null, omNode2Id);
+String omNode1RpcAddrKey = getOMAddrKeyWithSuffix(serviceID, omNode1Id);
+String omNode2RpcAddrKey = getOMAddrKeyWithSuffix(serviceID, omNode2Id);
Review comment:
As discussed offline for now removed Kerberos config.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 319898)
Time Spent: 5.5h (was: 5h 20m)
> Make Kerberos related configuration support HA style config
> ---
>
> Key: HDDS-2162
> URL: https://issues.apache.org/jira/browse/HDDS-2162
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5.5h
> Remaining Estimate: 0h
>
> To have a single configuration to use across OM cluster, few of the configs
> like
> OZONE_OM_KERBEROS_KEYTAB_FILE_KEY,
> OZONE_OM_KERBEROS_PRINCIPAL_KEY,
> OZONE_OM_HTTP_KERBEROS_KEYTAB_FILE,
> OZONE_OM_HTTP_KERBEROS_PRINCIPAL_KEY need to support configs which append
> with service id and node id.
>
> Addressed OM_DB_DIRS, OZONE_OM_ADDRESS_KEY also in this patch.
>
> This Jira is to fix the above configs.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=319331=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319331
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 27/Sep/19 03:42
Start Date: 27/Sep/19 03:42
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1511:
HDDS-2162. Make Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328903326
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##
@@ -309,13 +305,33 @@ private OzoneManager(OzoneConfiguration conf) throws
IOException,
AuthenticationException {
super(OzoneVersionInfo.OZONE_VERSION_INFO);
Preconditions.checkNotNull(conf);
-configuration = conf;
+configuration = new OzoneConfiguration(conf);
Review comment:
here configuration = new OzoneConfiguration(conf);
And later I am changing the configuration property values in this new object
so that it will not affect to original configuration.
I just checked with below:
```
OzoneConfiguration configuration = new OzoneConfiguration();
configuration.set(OZONE_METADATA_DIRS,
folder.newFolder().getAbsolutePath());
OzoneConfiguration configuration1 = new
OzoneConfiguration(configuration);
configuration1.set(OZONE_METADATA_DIRS, "bharat");
System.out.println(configuration.get(OZONE_METADATA_DIRS));
System.out.println(configuration1.get(OZONE_METADATA_DIRS));
```
/var/folders/g5/fk451xl14vdf891pq7b6m6v0gp/T/junit852875409842836/junit4506171024308775995
bharat
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 319331)
Time Spent: 5h 20m (was: 5h 10m)
> Make Kerberos related configuration support HA style config
> ---
>
> Key: HDDS-2162
> URL: https://issues.apache.org/jira/browse/HDDS-2162
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> To have a single configuration to use across OM cluster, few of the configs
> like
> OZONE_OM_KERBEROS_KEYTAB_FILE_KEY,
> OZONE_OM_KERBEROS_PRINCIPAL_KEY,
> OZONE_OM_HTTP_KERBEROS_KEYTAB_FILE,
> OZONE_OM_HTTP_KERBEROS_PRINCIPAL_KEY need to support configs which append
> with service id and node id.
>
> Addressed OM_DB_DIRS, OZONE_OM_ADDRESS_KEY also in this patch.
>
> This Jira is to fix the above configs.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=319324=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319324
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 27/Sep/19 02:57
Start Date: 27/Sep/19 02:57
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1511:
HDDS-2162. Make Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328897127
##
File path:
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerConfiguration.java
##
@@ -119,10 +119,13 @@ public void testDefaultPortIfNotSpecified() throws
Exception {
String omNode1Id = "omNode1";
String omNode2Id = "omNode2";
String omNodesKeyValue = omNode1Id + "," + omNode2Id;
-conf.set(OMConfigKeys.OZONE_OM_NODES_KEY, omNodesKeyValue);
+String serviceID = "service1";
+conf.set(OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY, serviceID);
+conf.set(OMConfigKeys.OZONE_OM_NODES_KEY + "." + serviceID,
+omNodesKeyValue);
-String omNode1RpcAddrKey = getOMAddrKeyWithSuffix(null, omNode1Id);
-String omNode2RpcAddrKey = getOMAddrKeyWithSuffix(null, omNode2Id);
+String omNode1RpcAddrKey = getOMAddrKeyWithSuffix(serviceID, omNode1Id);
+String omNode2RpcAddrKey = getOMAddrKeyWithSuffix(serviceID, omNode2Id);
Review comment:
Discussed offline, from my understanding this is being done as to share
config across all OM's. And this PR is not changing any config loading code of
OM HA, it just added Kerberos/DB config as described in Jira description.
Anu said we don't require it, and we shall continue the discussion later to
see how we can do it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 319324)
Time Spent: 5h (was: 4h 50m)
> Make Kerberos related configuration support HA style config
> ---
>
> Key: HDDS-2162
> URL: https://issues.apache.org/jira/browse/HDDS-2162
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h
> Remaining Estimate: 0h
>
> To have a single configuration to use across OM cluster, few of the configs
> like
> OZONE_OM_KERBEROS_KEYTAB_FILE_KEY,
> OZONE_OM_KERBEROS_PRINCIPAL_KEY,
> OZONE_OM_HTTP_KERBEROS_KEYTAB_FILE,
> OZONE_OM_HTTP_KERBEROS_PRINCIPAL_KEY need to support configs which append
> with service id and node id.
>
> Addressed OM_DB_DIRS, OZONE_OM_ADDRESS_KEY also in this patch.
>
> This Jira is to fix the above configs.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=319325=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319325
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 27/Sep/19 02:57
Start Date: 27/Sep/19 02:57
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1511:
HDDS-2162. Make Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328897127
##
File path:
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerConfiguration.java
##
@@ -119,10 +119,13 @@ public void testDefaultPortIfNotSpecified() throws
Exception {
String omNode1Id = "omNode1";
String omNode2Id = "omNode2";
String omNodesKeyValue = omNode1Id + "," + omNode2Id;
-conf.set(OMConfigKeys.OZONE_OM_NODES_KEY, omNodesKeyValue);
+String serviceID = "service1";
+conf.set(OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY, serviceID);
+conf.set(OMConfigKeys.OZONE_OM_NODES_KEY + "." + serviceID,
+omNodesKeyValue);
-String omNode1RpcAddrKey = getOMAddrKeyWithSuffix(null, omNode1Id);
-String omNode2RpcAddrKey = getOMAddrKeyWithSuffix(null, omNode2Id);
+String omNode1RpcAddrKey = getOMAddrKeyWithSuffix(serviceID, omNode1Id);
+String omNode2RpcAddrKey = getOMAddrKeyWithSuffix(serviceID, omNode2Id);
Review comment:
Discussed offline, from my understanding this is being done as to share
config across all OM's. And this PR is not changing any config loading code of
OM HA, it just added Kerberos/DB config as described in Jira description.
Anu said we don't require the current way, and we shall continue the
discussion later to see how we can do it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 319325)
Time Spent: 5h 10m (was: 5h)
> Make Kerberos related configuration support HA style config
> ---
>
> Key: HDDS-2162
> URL: https://issues.apache.org/jira/browse/HDDS-2162
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> To have a single configuration to use across OM cluster, few of the configs
> like
> OZONE_OM_KERBEROS_KEYTAB_FILE_KEY,
> OZONE_OM_KERBEROS_PRINCIPAL_KEY,
> OZONE_OM_HTTP_KERBEROS_KEYTAB_FILE,
> OZONE_OM_HTTP_KERBEROS_PRINCIPAL_KEY need to support configs which append
> with service id and node id.
>
> Addressed OM_DB_DIRS, OZONE_OM_ADDRESS_KEY also in this patch.
>
> This Jira is to fix the above configs.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=319317=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319317
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 27/Sep/19 02:08
Start Date: 27/Sep/19 02:08
Worklog Time Spent: 10m
Work Description: anuengineer commented on pull request #1511: HDDS-2162.
Make Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328890013
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##
@@ -309,13 +305,33 @@ private OzoneManager(OzoneConfiguration conf) throws
IOException,
AuthenticationException {
super(OzoneVersionInfo.OZONE_VERSION_INFO);
Preconditions.checkNotNull(conf);
-configuration = conf;
+configuration = new OzoneConfiguration(conf);
Review comment:
But you just lost the reference to the original object. I am slightly
confused here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 319317)
Time Spent: 4h 50m (was: 4h 40m)
> Make Kerberos related configuration support HA style config
> ---
>
> Key: HDDS-2162
> URL: https://issues.apache.org/jira/browse/HDDS-2162
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> To have a single configuration to use across OM cluster, few of the configs
> like
> OZONE_OM_KERBEROS_KEYTAB_FILE_KEY,
> OZONE_OM_KERBEROS_PRINCIPAL_KEY,
> OZONE_OM_HTTP_KERBEROS_KEYTAB_FILE,
> OZONE_OM_HTTP_KERBEROS_PRINCIPAL_KEY need to support configs which append
> with service id and node id.
>
> Addressed OM_DB_DIRS, OZONE_OM_ADDRESS_KEY also in this patch.
>
> This Jira is to fix the above configs.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=319316=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319316
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 27/Sep/19 02:06
Start Date: 27/Sep/19 02:06
Worklog Time Spent: 10m
Work Description: anuengineer commented on pull request #1511: HDDS-2162.
Make Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328889881
##
File path:
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerConfiguration.java
##
@@ -119,10 +119,13 @@ public void testDefaultPortIfNotSpecified() throws
Exception {
String omNode1Id = "omNode1";
String omNode2Id = "omNode2";
String omNodesKeyValue = omNode1Id + "," + omNode2Id;
-conf.set(OMConfigKeys.OZONE_OM_NODES_KEY, omNodesKeyValue);
+String serviceID = "service1";
+conf.set(OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY, serviceID);
+conf.set(OMConfigKeys.OZONE_OM_NODES_KEY + "." + serviceID,
+omNodesKeyValue);
-String omNode1RpcAddrKey = getOMAddrKeyWithSuffix(null, omNode1Id);
-String omNode2RpcAddrKey = getOMAddrKeyWithSuffix(null, omNode2Id);
+String omNode1RpcAddrKey = getOMAddrKeyWithSuffix(serviceID, omNode1Id);
+String omNode2RpcAddrKey = getOMAddrKeyWithSuffix(serviceID, omNode2Id);
Review comment:
> Not got your last part what is proposed.
https://docs.microsoft.com/en-us/windows/win32/ad/name-formats-for-unique-spns
OzoneManager/host1.example.com/CN=hrdb,OU=mktg,DC=example,DC=com
OzoneManager/host2.example.com/CN=hrdb,OU=mktg,DC=example,DC=com
OzoneManager/host3.example.com/CN=hrdb,OU=mktg,DC=example,DC=com
This is all we need, is what I am trying to say.
> Suppose the user wants to use different keytab file location/principal
name it will also help in this situation.
Why would you want separate identities to communicate to the same service ?
Can you give me an example of why this would be needed ? More over, why support
that identity via naming tricks in Ozone instead of creating an new SPN in
Kerberos Domain?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 319316)
Time Spent: 4h 40m (was: 4.5h)
> Make Kerberos related configuration support HA style config
> ---
>
> Key: HDDS-2162
> URL: https://issues.apache.org/jira/browse/HDDS-2162
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> To have a single configuration to use across OM cluster, few of the configs
> like
> OZONE_OM_KERBEROS_KEYTAB_FILE_KEY,
> OZONE_OM_KERBEROS_PRINCIPAL_KEY,
> OZONE_OM_HTTP_KERBEROS_KEYTAB_FILE,
> OZONE_OM_HTTP_KERBEROS_PRINCIPAL_KEY need to support configs which append
> with service id and node id.
>
> Addressed OM_DB_DIRS, OZONE_OM_ADDRESS_KEY also in this patch.
>
> This Jira is to fix the above configs.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=319270=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319270
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 26/Sep/19 23:13
Start Date: 26/Sep/19 23:13
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1511:
HDDS-2162. Make Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328861588
##
File path:
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerConfiguration.java
##
@@ -119,10 +119,13 @@ public void testDefaultPortIfNotSpecified() throws
Exception {
String omNode1Id = "omNode1";
String omNode2Id = "omNode2";
String omNodesKeyValue = omNode1Id + "," + omNode2Id;
-conf.set(OMConfigKeys.OZONE_OM_NODES_KEY, omNodesKeyValue);
+String serviceID = "service1";
+conf.set(OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY, serviceID);
+conf.set(OMConfigKeys.OZONE_OM_NODES_KEY + "." + serviceID,
+omNodesKeyValue);
-String omNode1RpcAddrKey = getOMAddrKeyWithSuffix(null, omNode1Id);
-String omNode2RpcAddrKey = getOMAddrKeyWithSuffix(null, omNode2Id);
+String omNode1RpcAddrKey = getOMAddrKeyWithSuffix(serviceID, omNode1Id);
+String omNode2RpcAddrKey = getOMAddrKeyWithSuffix(serviceID, omNode2Id);
Review comment:
> I have an uber comment on this JIRA. Under Ozone, what we really need is 3
+ 3 six kerberos Identites.
>
> Why don't we just follow the standard Kerberos SPN names? Simple take one
config key from the user, either the SPN or file name path the service kerberos
identity.
>
> Once you have this, we don't have to do any second guessing or munching of
names with any other strings -- after all it is just a service on a host. Code
is simpler, and the best part, it is simple enough for any one to understand.
In other words, what evil are we trying to prevent here with all the service
name munching ?
Not got your last part what is proposed.
This is done for Kerberos settings and also for other configs like OM DB
DIrs Http/Https Address. Suppose the user wants to use different keytab file
location/principal name it will also help in this situation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 319270)
Time Spent: 4.5h (was: 4h 20m)
> Make Kerberos related configuration support HA style config
> ---
>
> Key: HDDS-2162
> URL: https://issues.apache.org/jira/browse/HDDS-2162
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> To have a single configuration to use across OM cluster, few of the configs
> like
> OZONE_OM_KERBEROS_KEYTAB_FILE_KEY,
> OZONE_OM_KERBEROS_PRINCIPAL_KEY,
> OZONE_OM_HTTP_KERBEROS_KEYTAB_FILE,
> OZONE_OM_HTTP_KERBEROS_PRINCIPAL_KEY need to support configs which append
> with service id and node id.
>
> Addressed OM_DB_DIRS, OZONE_OM_ADDRESS_KEY also in this patch.
>
> This Jira is to fix the above configs.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=319268=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319268
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 26/Sep/19 23:12
Start Date: 26/Sep/19 23:12
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1511:
HDDS-2162. Make Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328861588
##
File path:
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerConfiguration.java
##
@@ -119,10 +119,13 @@ public void testDefaultPortIfNotSpecified() throws
Exception {
String omNode1Id = "omNode1";
String omNode2Id = "omNode2";
String omNodesKeyValue = omNode1Id + "," + omNode2Id;
-conf.set(OMConfigKeys.OZONE_OM_NODES_KEY, omNodesKeyValue);
+String serviceID = "service1";
+conf.set(OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY, serviceID);
+conf.set(OMConfigKeys.OZONE_OM_NODES_KEY + "." + serviceID,
+omNodesKeyValue);
-String omNode1RpcAddrKey = getOMAddrKeyWithSuffix(null, omNode1Id);
-String omNode2RpcAddrKey = getOMAddrKeyWithSuffix(null, omNode2Id);
+String omNode1RpcAddrKey = getOMAddrKeyWithSuffix(serviceID, omNode1Id);
+String omNode2RpcAddrKey = getOMAddrKeyWithSuffix(serviceID, omNode2Id);
Review comment:
> I have an uber comment on this JIRA. Under Ozone, what we really need is 3
+ 3 six kerberos Identites.
>
> Why don't we just follow the standard Kerberos SPN names? Simple take one
config key from the user, either the SPN or file name path the service kerberos
identity.
>
> Once you have this, we don't have to do any second guessing or munching of
names with any other strings -- after all it is just a service on a host. Code
is simpler, and the best part, it is simple enough for any one to understand.
In other words, what evil are we trying to prevent here with all the service
name munching ?
This is done for Kerberos settings and also for other configs like OM DB
DIrs Http/Https Address. Suppose the user wants to use different keytab file
location/principal name it will also help in this situation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 319268)
Time Spent: 4h 20m (was: 4h 10m)
> Make Kerberos related configuration support HA style config
> ---
>
> Key: HDDS-2162
> URL: https://issues.apache.org/jira/browse/HDDS-2162
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> To have a single configuration to use across OM cluster, few of the configs
> like
> OZONE_OM_KERBEROS_KEYTAB_FILE_KEY,
> OZONE_OM_KERBEROS_PRINCIPAL_KEY,
> OZONE_OM_HTTP_KERBEROS_KEYTAB_FILE,
> OZONE_OM_HTTP_KERBEROS_PRINCIPAL_KEY need to support configs which append
> with service id and node id.
>
> Addressed OM_DB_DIRS, OZONE_OM_ADDRESS_KEY also in this patch.
>
> This Jira is to fix the above configs.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=319267=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319267
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 26/Sep/19 23:11
Start Date: 26/Sep/19 23:11
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1511:
HDDS-2162. Make Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328861588
##
File path:
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerConfiguration.java
##
@@ -119,10 +119,13 @@ public void testDefaultPortIfNotSpecified() throws
Exception {
String omNode1Id = "omNode1";
String omNode2Id = "omNode2";
String omNodesKeyValue = omNode1Id + "," + omNode2Id;
-conf.set(OMConfigKeys.OZONE_OM_NODES_KEY, omNodesKeyValue);
+String serviceID = "service1";
+conf.set(OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY, serviceID);
+conf.set(OMConfigKeys.OZONE_OM_NODES_KEY + "." + serviceID,
+omNodesKeyValue);
-String omNode1RpcAddrKey = getOMAddrKeyWithSuffix(null, omNode1Id);
-String omNode2RpcAddrKey = getOMAddrKeyWithSuffix(null, omNode2Id);
+String omNode1RpcAddrKey = getOMAddrKeyWithSuffix(serviceID, omNode1Id);
+String omNode2RpcAddrKey = getOMAddrKeyWithSuffix(serviceID, omNode2Id);
Review comment:
> I have an uber comment on this JIRA. Under Ozone, what we really need is 3
+ 3 six kerberos Identites.
>
> Why don't we just follow the standard Kerberos SPN names? Simple take one
config key from the user, either the SPN or file name path the service kerberos
identity.
>
> Once you have this, we don't have to do any second guessing or munching of
names with any other strings -- after all it is just a service on a host. Code
is simpler, and the best part, it is simple enough for any one to understand.
In other words, what evil are we trying to prevent here with all the service
name munching ?
I have an uber comment on this JIRA. Under Ozone, what we really need is 3 +
3 six kerberos Identites.
Why don't we just follow the standard Kerberos SPN names? Simple take one
config key from the user, either the SPN or file name path the service kerberos
identity.
Once you have this, we don't have to do any second guessing or munching of
names with any other strings -- after all it is just a service on a host. Code
is simpler, and the best part, it is simple enough for any one to understand.
In other words, what evil are we trying to prevent here with all the service
name munching ?
This is done for Kerberos settings and also for other configs like OM DB
DIrs Http/Https Address. Suppose the user wants to use different keytab file
location/principal name it will also help in this situation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 319267)
Time Spent: 4h 10m (was: 4h)
> Make Kerberos related configuration support HA style config
> ---
>
> Key: HDDS-2162
> URL: https://issues.apache.org/jira/browse/HDDS-2162
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> To have a single configuration to use across OM cluster, few of the configs
> like
> OZONE_OM_KERBEROS_KEYTAB_FILE_KEY,
> OZONE_OM_KERBEROS_PRINCIPAL_KEY,
> OZONE_OM_HTTP_KERBEROS_KEYTAB_FILE,
> OZONE_OM_HTTP_KERBEROS_PRINCIPAL_KEY need to support configs which append
> with service id and node id.
>
> Addressed OM_DB_DIRS, OZONE_OM_ADDRESS_KEY also in this patch.
>
> This Jira is to fix the above configs.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=319265=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319265
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 26/Sep/19 23:08
Start Date: 26/Sep/19 23:08
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1511:
HDDS-2162. Make Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328860862
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##
@@ -309,13 +305,33 @@ private OzoneManager(OzoneConfiguration conf) throws
IOException,
AuthenticationException {
super(OzoneVersionInfo.OZONE_VERSION_INFO);
Preconditions.checkNotNull(conf);
-configuration = conf;
+configuration = new OzoneConfiguration(conf);
Review comment:
This is being done, so not to change the original values of configuration
object passed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 319265)
Time Spent: 4h (was: 3h 50m)
> Make Kerberos related configuration support HA style config
> ---
>
> Key: HDDS-2162
> URL: https://issues.apache.org/jira/browse/HDDS-2162
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h
> Remaining Estimate: 0h
>
> To have a single configuration to use across OM cluster, few of the configs
> like
> OZONE_OM_KERBEROS_KEYTAB_FILE_KEY,
> OZONE_OM_KERBEROS_PRINCIPAL_KEY,
> OZONE_OM_HTTP_KERBEROS_KEYTAB_FILE,
> OZONE_OM_HTTP_KERBEROS_PRINCIPAL_KEY need to support configs which append
> with service id and node id.
>
> Addressed OM_DB_DIRS, OZONE_OM_ADDRESS_KEY also in this patch.
>
> This Jira is to fix the above configs.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=319261=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319261
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 26/Sep/19 22:35
Start Date: 26/Sep/19 22:35
Worklog Time Spent: 10m
Work Description: anuengineer commented on pull request #1511: HDDS-2162.
Make Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328839876
##
File path:
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerConfiguration.java
##
@@ -119,10 +119,13 @@ public void testDefaultPortIfNotSpecified() throws
Exception {
String omNode1Id = "omNode1";
String omNode2Id = "omNode2";
String omNodesKeyValue = omNode1Id + "," + omNode2Id;
-conf.set(OMConfigKeys.OZONE_OM_NODES_KEY, omNodesKeyValue);
+String serviceID = "service1";
+conf.set(OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY, serviceID);
+conf.set(OMConfigKeys.OZONE_OM_NODES_KEY + "." + serviceID,
+omNodesKeyValue);
-String omNode1RpcAddrKey = getOMAddrKeyWithSuffix(null, omNode1Id);
-String omNode2RpcAddrKey = getOMAddrKeyWithSuffix(null, omNode2Id);
+String omNode1RpcAddrKey = getOMAddrKeyWithSuffix(serviceID, omNode1Id);
+String omNode2RpcAddrKey = getOMAddrKeyWithSuffix(serviceID, omNode2Id);
Review comment:
Don't need to verify that these strings are in the expected format here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 319261)
Time Spent: 3h 50m (was: 3h 40m)
> Make Kerberos related configuration support HA style config
> ---
>
> Key: HDDS-2162
> URL: https://issues.apache.org/jira/browse/HDDS-2162
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> To have a single configuration to use across OM cluster, few of the configs
> like
> OZONE_OM_KERBEROS_KEYTAB_FILE_KEY,
> OZONE_OM_KERBEROS_PRINCIPAL_KEY,
> OZONE_OM_HTTP_KERBEROS_KEYTAB_FILE,
> OZONE_OM_HTTP_KERBEROS_PRINCIPAL_KEY need to support configs which append
> with service id and node id.
>
> Addressed OM_DB_DIRS, OZONE_OM_ADDRESS_KEY also in this patch.
>
> This Jira is to fix the above configs.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=319260=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319260
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 26/Sep/19 22:35
Start Date: 26/Sep/19 22:35
Worklog Time Spent: 10m
Work Description: anuengineer commented on pull request #1511: HDDS-2162.
Make Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328851483
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##
@@ -309,13 +305,33 @@ private OzoneManager(OzoneConfiguration conf) throws
IOException,
AuthenticationException {
super(OzoneVersionInfo.OZONE_VERSION_INFO);
Preconditions.checkNotNull(conf);
-configuration = conf;
+configuration = new OzoneConfiguration(conf);
Review comment:
This function passes a conf object of type OzoneConfiguration. Why are we
reallocating a new object before assigining that configuration obect to the
member variable?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 319260)
Time Spent: 3h 40m (was: 3.5h)
> Make Kerberos related configuration support HA style config
> ---
>
> Key: HDDS-2162
> URL: https://issues.apache.org/jira/browse/HDDS-2162
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> To have a single configuration to use across OM cluster, few of the configs
> like
> OZONE_OM_KERBEROS_KEYTAB_FILE_KEY,
> OZONE_OM_KERBEROS_PRINCIPAL_KEY,
> OZONE_OM_HTTP_KERBEROS_KEYTAB_FILE,
> OZONE_OM_HTTP_KERBEROS_PRINCIPAL_KEY need to support configs which append
> with service id and node id.
>
> Addressed OM_DB_DIRS, OZONE_OM_ADDRESS_KEY also in this patch.
>
> This Jira is to fix the above configs.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[ https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=319127=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319127 ] ASF GitHub Bot logged work on HDDS-2162: Author: ASF GitHub Bot Created on: 26/Sep/19 18:21 Start Date: 26/Sep/19 18:21 Worklog Time Spent: 10m Work Description: arp7 commented on issue #1511: HDDS-2162. Make Kerberos related configuration support HA style config. URL: https://github.com/apache/hadoop/pull/1511#issuecomment-535626926 /retest This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 319127) Time Spent: 3.5h (was: 3h 20m) > Make Kerberos related configuration support HA style config > --- > > Key: HDDS-2162 > URL: https://issues.apache.org/jira/browse/HDDS-2162 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Time Spent: 3.5h > Remaining Estimate: 0h > > To have a single configuration to use across OM cluster, few of the configs > like > OZONE_OM_KERBEROS_KEYTAB_FILE_KEY, > OZONE_OM_KERBEROS_PRINCIPAL_KEY, > OZONE_OM_HTTP_KERBEROS_KEYTAB_FILE, > OZONE_OM_HTTP_KERBEROS_PRINCIPAL_KEY need to support configs which append > with service id and node id. > > Addressed OM_DB_DIRS, OZONE_OM_ADDRESS_KEY also in this patch. > > This Jira is to fix the above configs. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[ https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=319038=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319038 ] ASF GitHub Bot logged work on HDDS-2162: Author: ASF GitHub Bot Created on: 26/Sep/19 15:21 Start Date: 26/Sep/19 15:21 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on issue #1511: HDDS-2162. Make Kerberos related configuration support HA style config. URL: https://github.com/apache/hadoop/pull/1511#issuecomment-53194 /retest This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 319038) Time Spent: 3h 20m (was: 3h 10m) > Make Kerberos related configuration support HA style config > --- > > Key: HDDS-2162 > URL: https://issues.apache.org/jira/browse/HDDS-2162 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Time Spent: 3h 20m > Remaining Estimate: 0h > > To have a single configuration to use across OM cluster, few of the configs > like > OZONE_OM_KERBEROS_KEYTAB_FILE_KEY, > OZONE_OM_KERBEROS_PRINCIPAL_KEY, > OZONE_OM_HTTP_KERBEROS_KEYTAB_FILE, > OZONE_OM_HTTP_KERBEROS_PRINCIPAL_KEY need to support configs which append > with service id and node id. > > Addressed OM_DB_DIRS, OZONE_OM_ADDRESS_KEY also in this patch. > > This Jira is to fix the above configs. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[ https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=318764=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318764 ] ASF GitHub Bot logged work on HDDS-2162: Author: ASF GitHub Bot Created on: 26/Sep/19 04:00 Start Date: 26/Sep/19 04:00 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on issue #1511: HDDS-2162. Make Kerberos related configuration support HA style config. URL: https://github.com/apache/hadoop/pull/1511#issuecomment-535322831 /retest This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 318764) Time Spent: 3h 10m (was: 3h) > Make Kerberos related configuration support HA style config > --- > > Key: HDDS-2162 > URL: https://issues.apache.org/jira/browse/HDDS-2162 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > To have a single configuration to use across OM cluster, few of the configs > like > OZONE_OM_KERBEROS_KEYTAB_FILE_KEY, > OZONE_OM_KERBEROS_PRINCIPAL_KEY, > OZONE_OM_HTTP_KERBEROS_KEYTAB_FILE, > OZONE_OM_HTTP_KERBEROS_PRINCIPAL_KEY need to support configs which append > with service id and node id. > > Addressed OM_DB_DIRS, OZONE_OM_ADDRESS_KEY also in this patch. > > This Jira is to fix the above configs. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[ https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=318659=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318659 ] ASF GitHub Bot logged work on HDDS-2162: Author: ASF GitHub Bot Created on: 25/Sep/19 23:29 Start Date: 25/Sep/19 23:29 Worklog Time Spent: 10m Work Description: arp7 commented on issue #1511: HDDS-2162. Make Kerberos related configuration support HA style config. URL: https://github.com/apache/hadoop/pull/1511#issuecomment-535264311 /retest This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 318659) Time Spent: 3h (was: 2h 50m) > Make Kerberos related configuration support HA style config > --- > > Key: HDDS-2162 > URL: https://issues.apache.org/jira/browse/HDDS-2162 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Time Spent: 3h > Remaining Estimate: 0h > > To have a single configuration to use across OM cluster, few of the configs > like > OZONE_OM_KERBEROS_KEYTAB_FILE_KEY, > OZONE_OM_KERBEROS_PRINCIPAL_KEY, > OZONE_OM_HTTP_KERBEROS_KEYTAB_FILE, > OZONE_OM_HTTP_KERBEROS_PRINCIPAL_KEY need to support configs which append > with service id and node id. > > Addressed OM_DB_DIRS, OZONE_OM_ADDRESS_KEY also in this patch. > > This Jira is to fix the above configs. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=318657=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318657
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 25/Sep/19 23:26
Start Date: 25/Sep/19 23:26
Worklog Time Spent: 10m
Work Description: arp7 commented on pull request #1511: HDDS-2162. Make
Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328380337
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ha/OMHANodeDetails.java
##
@@ -0,0 +1,309 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.ha;
+
+import com.google.common.base.Preconditions;
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.OzoneConsts;
+import org.apache.hadoop.ozone.OzoneIllegalArgumentException;
+import org.apache.hadoop.ozone.om.OMConfigKeys;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_ADDRESS_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODES_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODE_ID_KEY;
+import static
org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_DEFAULT;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY;
+
+/**
+ * Class which maintains peer information and it's own OM node information.
+ */
+public class OMHANodeDetails {
+
+ public static final Logger LOG =
+ LoggerFactory.getLogger(OMHANodeDetails.class);
+ private final OMNodeDetails localNodeDetails;
+ private final List peerNodeDetails;
+
+ public OMHANodeDetails(OMNodeDetails localNodeDetails,
+ List peerNodeDetails) {
+this.localNodeDetails = localNodeDetails;
+this.peerNodeDetails = peerNodeDetails;
+ }
+
+ public OMNodeDetails getLocalNodeDetails() {
+return localNodeDetails;
+ }
+
+ public List< OMNodeDetails > getPeerNodeDetails() {
+return peerNodeDetails;
+ }
+
+
+ /**
+ * Inspects and loads OM node configurations.
+ *
+ * If {@link OMConfigKeys#OZONE_OM_SERVICE_IDS_KEY} is configured with
+ * multiple ids and/ or if {@link OMConfigKeys#OZONE_OM_NODE_ID_KEY} is not
+ * specifically configured , this method determines the omServiceId
+ * and omNodeId by matching the node's address with the configured
+ * addresses. When a match is found, it sets the omServicId and omNodeId from
+ * the corresponding configuration key. This method also finds the OM peers
+ * nodes belonging to the same OM service.
+ *
+ * @param conf
+ */
+ public static OMHANodeDetails loadOMHAConfig(OzoneConfiguration conf) {
+InetSocketAddress localRpcAddress = null;
+String localOMServiceId = null;
+String localOMNodeId = null;
+int localRatisPort = 0;
+Collection omServiceIds = conf.getTrimmedStringCollection(
+OZONE_OM_SERVICE_IDS_KEY);
+
+String knownOMNodeId = conf.get(OZONE_OM_NODE_ID_KEY);
Review comment:
Can you file a jira to add that?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 318657)
Time Spent: 2h 50m (was: 2h 40m)
> Make Kerberos related configuration support HA style config
> ---
>
> Key: HDDS-2162
> URL: https://issues.apache.org/jira/browse/HDDS-2162
>
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[ https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=318607=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318607 ] ASF GitHub Bot logged work on HDDS-2162: Author: ASF GitHub Bot Created on: 25/Sep/19 21:18 Start Date: 25/Sep/19 21:18 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1511: HDDS-2162. Make Kerberos related configuration support HA style config. URL: https://github.com/apache/hadoop/pull/1511#issuecomment-535225523 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 39 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 13 | Maven dependency ordering for branch | | -1 | mvninstall | 30 | hadoop-hdds in trunk failed. | | -1 | mvninstall | 23 | hadoop-ozone in trunk failed. | | -1 | compile | 22 | hadoop-hdds in trunk failed. | | -1 | compile | 15 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 52 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 994 | branch has no errors when building and testing our client artifacts. | | -1 | javadoc | 18 | hadoop-hdds in trunk failed. | | -1 | javadoc | 15 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 1073 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 27 | hadoop-hdds in trunk failed. | | -1 | findbugs | 15 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | 0 | mvndep | 13 | Maven dependency ordering for patch | | -1 | mvninstall | 30 | hadoop-hdds in the patch failed. | | -1 | mvninstall | 24 | hadoop-ozone in the patch failed. | | -1 | compile | 19 | hadoop-hdds in the patch failed. | | -1 | compile | 14 | hadoop-ozone in the patch failed. | | -1 | javac | 19 | hadoop-hdds in the patch failed. | | -1 | javac | 14 | hadoop-ozone in the patch failed. | | +1 | checkstyle | 49 | the patch passed | | +1 | mvnsite | 1 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 771 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 18 | hadoop-hdds in the patch failed. | | -1 | javadoc | 15 | hadoop-ozone in the patch failed. | | -1 | findbugs | 27 | hadoop-hdds in the patch failed. | | -1 | findbugs | 16 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | -1 | unit | 23 | hadoop-hdds in the patch failed. | | -1 | unit | 19 | hadoop-ozone in the patch failed. | | +1 | asflicense | 28 | The patch does not generate ASF License warnings. | | | | 2450 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1511 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 3bb453cc6bdc 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / bdaaa3b | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/3/artifact/out/branch-mvninstall-hadoop-hdds.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/3/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/3/artifact/out/branch-compile-hadoop-hdds.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/3/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/3/artifact/out/branch-javadoc-hadoop-hdds.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/3/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/3/artifact/out/branch-findbugs-hadoop-hdds.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/3/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/3/artifact/out/patch-mvninstall-hadoop-hdds.txt | | mvninstall |
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[ https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=318591=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318591 ] ASF GitHub Bot logged work on HDDS-2162: Author: ASF GitHub Bot Created on: 25/Sep/19 20:43 Start Date: 25/Sep/19 20:43 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on issue #1511: HDDS-2162. Make Kerberos related configuration support HA style config. URL: https://github.com/apache/hadoop/pull/1511#issuecomment-535212610 Thank You @arp7 for the review. I have addressed the review comments. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 318591) Time Spent: 2.5h (was: 2h 20m) > Make Kerberos related configuration support HA style config > --- > > Key: HDDS-2162 > URL: https://issues.apache.org/jira/browse/HDDS-2162 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > To have a single configuration to use across OM cluster, few of the configs > like > OZONE_OM_KERBEROS_KEYTAB_FILE_KEY, > OZONE_OM_KERBEROS_PRINCIPAL_KEY, > OZONE_OM_HTTP_KERBEROS_KEYTAB_FILE, > OZONE_OM_HTTP_KERBEROS_PRINCIPAL_KEY need to support configs which append > with service id and node id. > > Addressed OM_DB_DIRS, OZONE_OM_ADDRESS_KEY also in this patch. > > This Jira is to fix the above configs. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=318587=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318587
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 25/Sep/19 20:36
Start Date: 25/Sep/19 20:36
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1511:
HDDS-2162. Make Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328328247
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ha/OMHANodeDetails.java
##
@@ -0,0 +1,309 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.ha;
+
+import com.google.common.base.Preconditions;
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.OzoneConsts;
+import org.apache.hadoop.ozone.OzoneIllegalArgumentException;
+import org.apache.hadoop.ozone.om.OMConfigKeys;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_ADDRESS_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODES_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODE_ID_KEY;
+import static
org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_DEFAULT;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY;
+
+/**
+ * Class which maintains peer information and it's own OM node information.
+ */
+public class OMHANodeDetails {
+
+ public static final Logger LOG =
+ LoggerFactory.getLogger(OMHANodeDetails.class);
+ private final OMNodeDetails localNodeDetails;
+ private final List peerNodeDetails;
+
+ public OMHANodeDetails(OMNodeDetails localNodeDetails,
+ List peerNodeDetails) {
+this.localNodeDetails = localNodeDetails;
+this.peerNodeDetails = peerNodeDetails;
+ }
+
+ public OMNodeDetails getLocalNodeDetails() {
+return localNodeDetails;
+ }
+
+ public List< OMNodeDetails > getPeerNodeDetails() {
+return peerNodeDetails;
+ }
+
+
+ /**
+ * Inspects and loads OM node configurations.
+ *
+ * If {@link OMConfigKeys#OZONE_OM_SERVICE_IDS_KEY} is configured with
+ * multiple ids and/ or if {@link OMConfigKeys#OZONE_OM_NODE_ID_KEY} is not
+ * specifically configured , this method determines the omServiceId
+ * and omNodeId by matching the node's address with the configured
+ * addresses. When a match is found, it sets the omServicId and omNodeId from
+ * the corresponding configuration key. This method also finds the OM peers
+ * nodes belonging to the same OM service.
+ *
+ * @param conf
+ */
+ public static OMHANodeDetails loadOMHAConfig(OzoneConfiguration conf) {
+InetSocketAddress localRpcAddress = null;
+String localOMServiceId = null;
+String localOMNodeId = null;
+int localRatisPort = 0;
+Collection omServiceIds = conf.getTrimmedStringCollection(
+OZONE_OM_SERVICE_IDS_KEY);
+
+String knownOMNodeId = conf.get(OZONE_OM_NODE_ID_KEY);
+int found = 0;
+boolean isOMAddressSet = false;
+
+for (String serviceId : omServiceIds) {
+ Collection omNodeIds = OmUtils.getOMNodeIds(conf, serviceId);
+
+ if (omNodeIds.size() == 0) {
+String msg = "Configuration does not have any value set for " +
+OZONE_OM_NODES_KEY + " for service ID " + serviceId + ". List of "
+
+"OM Node ID's should be specified for the service ID";
+throw new OzoneIllegalArgumentException(msg);
+ }
+
+ List peerNodesList = new ArrayList<>();
+ boolean isPeer;
+ for (String nodeId : omNodeIds) {
+if (knownOMNodeId != null && !knownOMNodeId.equals(nodeId)) {
+ isPeer = true;
+} else {
+
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=318585=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318585
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 25/Sep/19 20:36
Start Date: 25/Sep/19 20:36
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1511:
HDDS-2162. Make Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328328143
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ha/OMHANodeDetails.java
##
@@ -0,0 +1,309 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.ha;
+
+import com.google.common.base.Preconditions;
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.OzoneConsts;
+import org.apache.hadoop.ozone.OzoneIllegalArgumentException;
+import org.apache.hadoop.ozone.om.OMConfigKeys;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_ADDRESS_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODES_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODE_ID_KEY;
+import static
org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_DEFAULT;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY;
+
+/**
+ * Class which maintains peer information and it's own OM node information.
+ */
+public class OMHANodeDetails {
+
+ public static final Logger LOG =
+ LoggerFactory.getLogger(OMHANodeDetails.class);
+ private final OMNodeDetails localNodeDetails;
+ private final List peerNodeDetails;
+
+ public OMHANodeDetails(OMNodeDetails localNodeDetails,
+ List peerNodeDetails) {
+this.localNodeDetails = localNodeDetails;
+this.peerNodeDetails = peerNodeDetails;
+ }
+
+ public OMNodeDetails getLocalNodeDetails() {
+return localNodeDetails;
+ }
+
+ public List< OMNodeDetails > getPeerNodeDetails() {
+return peerNodeDetails;
+ }
+
+
+ /**
+ * Inspects and loads OM node configurations.
+ *
+ * If {@link OMConfigKeys#OZONE_OM_SERVICE_IDS_KEY} is configured with
+ * multiple ids and/ or if {@link OMConfigKeys#OZONE_OM_NODE_ID_KEY} is not
+ * specifically configured , this method determines the omServiceId
+ * and omNodeId by matching the node's address with the configured
+ * addresses. When a match is found, it sets the omServicId and omNodeId from
+ * the corresponding configuration key. This method also finds the OM peers
+ * nodes belonging to the same OM service.
+ *
+ * @param conf
+ */
+ public static OMHANodeDetails loadOMHAConfig(OzoneConfiguration conf) {
+InetSocketAddress localRpcAddress = null;
+String localOMServiceId = null;
+String localOMNodeId = null;
+int localRatisPort = 0;
+Collection omServiceIds = conf.getTrimmedStringCollection(
+OZONE_OM_SERVICE_IDS_KEY);
+
+String knownOMNodeId = conf.get(OZONE_OM_NODE_ID_KEY);
+int found = 0;
+boolean isOMAddressSet = false;
+
+for (String serviceId : omServiceIds) {
+ Collection omNodeIds = OmUtils.getOMNodeIds(conf, serviceId);
+
+ if (omNodeIds.size() == 0) {
+String msg = "Configuration does not have any value set for " +
+OZONE_OM_NODES_KEY + " for service ID " + serviceId + ". List of "
+
+"OM Node ID's should be specified for the service ID";
+throw new OzoneIllegalArgumentException(msg);
+ }
+
+ List peerNodesList = new ArrayList<>();
+ boolean isPeer;
+ for (String nodeId : omNodeIds) {
+if (knownOMNodeId != null && !knownOMNodeId.equals(nodeId)) {
+ isPeer = true;
+} else {
+
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=318583=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318583
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 25/Sep/19 20:35
Start Date: 25/Sep/19 20:35
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1511:
HDDS-2162. Make Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328327822
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ha/OMHANodeDetails.java
##
@@ -0,0 +1,309 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.ha;
+
+import com.google.common.base.Preconditions;
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.OzoneConsts;
+import org.apache.hadoop.ozone.OzoneIllegalArgumentException;
+import org.apache.hadoop.ozone.om.OMConfigKeys;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_ADDRESS_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODES_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODE_ID_KEY;
+import static
org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_DEFAULT;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY;
+
+/**
+ * Class which maintains peer information and it's own OM node information.
+ */
+public class OMHANodeDetails {
+
+ public static final Logger LOG =
+ LoggerFactory.getLogger(OMHANodeDetails.class);
+ private final OMNodeDetails localNodeDetails;
+ private final List peerNodeDetails;
+
+ public OMHANodeDetails(OMNodeDetails localNodeDetails,
+ List peerNodeDetails) {
+this.localNodeDetails = localNodeDetails;
+this.peerNodeDetails = peerNodeDetails;
+ }
+
+ public OMNodeDetails getLocalNodeDetails() {
+return localNodeDetails;
+ }
+
+ public List< OMNodeDetails > getPeerNodeDetails() {
+return peerNodeDetails;
+ }
+
+
+ /**
+ * Inspects and loads OM node configurations.
+ *
+ * If {@link OMConfigKeys#OZONE_OM_SERVICE_IDS_KEY} is configured with
+ * multiple ids and/ or if {@link OMConfigKeys#OZONE_OM_NODE_ID_KEY} is not
+ * specifically configured , this method determines the omServiceId
+ * and omNodeId by matching the node's address with the configured
+ * addresses. When a match is found, it sets the omServicId and omNodeId from
+ * the corresponding configuration key. This method also finds the OM peers
+ * nodes belonging to the same OM service.
+ *
+ * @param conf
+ */
+ public static OMHANodeDetails loadOMHAConfig(OzoneConfiguration conf) {
+InetSocketAddress localRpcAddress = null;
+String localOMServiceId = null;
+String localOMNodeId = null;
+int localRatisPort = 0;
+Collection omServiceIds = conf.getTrimmedStringCollection(
+OZONE_OM_SERVICE_IDS_KEY);
+
+String knownOMNodeId = conf.get(OZONE_OM_NODE_ID_KEY);
Review comment:
This is generally used in MiniOzoneClusterHA testing. For each OM this is
set with different value. (As now we don't have federated OM setup it is not
really required at this point, I agree we need this at some point for testing)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 318583)
Time Spent: 2h (was: 1h 50m)
> Make Kerberos related configuration
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=318580=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318580
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 25/Sep/19 20:32
Start Date: 25/Sep/19 20:32
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1511:
HDDS-2162. Make Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328326246
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ha/OMHANodeDetails.java
##
@@ -0,0 +1,309 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.ha;
+
+import com.google.common.base.Preconditions;
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.OzoneConsts;
+import org.apache.hadoop.ozone.OzoneIllegalArgumentException;
+import org.apache.hadoop.ozone.om.OMConfigKeys;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_ADDRESS_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODES_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODE_ID_KEY;
+import static
org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_DEFAULT;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY;
+
+/**
+ * Class which maintains peer information and it's own OM node information.
+ */
+public class OMHANodeDetails {
+
+ public static final Logger LOG =
+ LoggerFactory.getLogger(OMHANodeDetails.class);
+ private final OMNodeDetails localNodeDetails;
+ private final List peerNodeDetails;
+
+ public OMHANodeDetails(OMNodeDetails localNodeDetails,
+ List peerNodeDetails) {
+this.localNodeDetails = localNodeDetails;
+this.peerNodeDetails = peerNodeDetails;
+ }
+
+ public OMNodeDetails getLocalNodeDetails() {
+return localNodeDetails;
+ }
+
+ public List< OMNodeDetails > getPeerNodeDetails() {
+return peerNodeDetails;
+ }
+
+
+ /**
+ * Inspects and loads OM node configurations.
+ *
+ * If {@link OMConfigKeys#OZONE_OM_SERVICE_IDS_KEY} is configured with
+ * multiple ids and/ or if {@link OMConfigKeys#OZONE_OM_NODE_ID_KEY} is not
+ * specifically configured , this method determines the omServiceId
+ * and omNodeId by matching the node's address with the configured
+ * addresses. When a match is found, it sets the omServicId and omNodeId from
+ * the corresponding configuration key. This method also finds the OM peers
+ * nodes belonging to the same OM service.
+ *
+ * @param conf
+ */
+ public static OMHANodeDetails loadOMHAConfig(OzoneConfiguration conf) {
+InetSocketAddress localRpcAddress = null;
+String localOMServiceId = null;
+String localOMNodeId = null;
+int localRatisPort = 0;
+Collection omServiceIds = conf.getTrimmedStringCollection(
+OZONE_OM_SERVICE_IDS_KEY);
+
+String knownOMNodeId = conf.get(OZONE_OM_NODE_ID_KEY);
+int found = 0;
+boolean isOMAddressSet = false;
+
+for (String serviceId : omServiceIds) {
+ Collection omNodeIds = OmUtils.getOMNodeIds(conf, serviceId);
+
+ if (omNodeIds.size() == 0) {
+String msg = "Configuration does not have any value set for " +
+OZONE_OM_NODES_KEY + " for service ID " + serviceId + ". List of "
+
+"OM Node ID's should be specified for the service ID";
+throw new OzoneIllegalArgumentException(msg);
+ }
+
+ List peerNodesList = new ArrayList<>();
+ boolean isPeer;
+ for (String nodeId : omNodeIds) {
+if (knownOMNodeId != null && !knownOMNodeId.equals(nodeId)) {
+ isPeer = true;
+} else {
+
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=318567=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318567
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 25/Sep/19 20:09
Start Date: 25/Sep/19 20:09
Worklog Time Spent: 10m
Work Description: arp7 commented on pull request #1511: HDDS-2162. Make
Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328316564
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ha/OMHANodeDetails.java
##
@@ -0,0 +1,309 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.ha;
+
+import com.google.common.base.Preconditions;
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.OzoneConsts;
+import org.apache.hadoop.ozone.OzoneIllegalArgumentException;
+import org.apache.hadoop.ozone.om.OMConfigKeys;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_ADDRESS_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODES_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODE_ID_KEY;
+import static
org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_DEFAULT;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY;
+
+/**
+ * Class which maintains peer information and it's own OM node information.
+ */
+public class OMHANodeDetails {
+
+ public static final Logger LOG =
+ LoggerFactory.getLogger(OMHANodeDetails.class);
+ private final OMNodeDetails localNodeDetails;
+ private final List peerNodeDetails;
+
+ public OMHANodeDetails(OMNodeDetails localNodeDetails,
+ List peerNodeDetails) {
+this.localNodeDetails = localNodeDetails;
+this.peerNodeDetails = peerNodeDetails;
+ }
+
+ public OMNodeDetails getLocalNodeDetails() {
+return localNodeDetails;
+ }
+
+ public List< OMNodeDetails > getPeerNodeDetails() {
+return peerNodeDetails;
+ }
+
+
+ /**
+ * Inspects and loads OM node configurations.
+ *
+ * If {@link OMConfigKeys#OZONE_OM_SERVICE_IDS_KEY} is configured with
+ * multiple ids and/ or if {@link OMConfigKeys#OZONE_OM_NODE_ID_KEY} is not
+ * specifically configured , this method determines the omServiceId
+ * and omNodeId by matching the node's address with the configured
+ * addresses. When a match is found, it sets the omServicId and omNodeId from
+ * the corresponding configuration key. This method also finds the OM peers
+ * nodes belonging to the same OM service.
+ *
+ * @param conf
+ */
+ public static OMHANodeDetails loadOMHAConfig(OzoneConfiguration conf) {
+InetSocketAddress localRpcAddress = null;
+String localOMServiceId = null;
+String localOMNodeId = null;
+int localRatisPort = 0;
+Collection omServiceIds = conf.getTrimmedStringCollection(
+OZONE_OM_SERVICE_IDS_KEY);
+
+String knownOMNodeId = conf.get(OZONE_OM_NODE_ID_KEY);
Review comment:
Oh there should be one then. We cannot assume nodeID is unique across
services. We can do so in a separate jira.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 318567)
Time Spent: 1h 40m (was: 1.5h)
> Make Kerberos related configuration support HA style config
> ---
>
> Key: HDDS-2162
>
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=318561=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318561
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 25/Sep/19 20:06
Start Date: 25/Sep/19 20:06
Worklog Time Spent: 10m
Work Description: arp7 commented on pull request #1511: HDDS-2162. Make
Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328315566
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ha/OMHANodeDetails.java
##
@@ -0,0 +1,309 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.ha;
+
+import com.google.common.base.Preconditions;
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.OzoneConsts;
+import org.apache.hadoop.ozone.OzoneIllegalArgumentException;
+import org.apache.hadoop.ozone.om.OMConfigKeys;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_ADDRESS_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODES_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODE_ID_KEY;
+import static
org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_DEFAULT;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY;
+
+/**
+ * Class which maintains peer information and it's own OM node information.
+ */
+public class OMHANodeDetails {
+
+ public static final Logger LOG =
+ LoggerFactory.getLogger(OMHANodeDetails.class);
+ private final OMNodeDetails localNodeDetails;
+ private final List peerNodeDetails;
+
+ public OMHANodeDetails(OMNodeDetails localNodeDetails,
+ List peerNodeDetails) {
+this.localNodeDetails = localNodeDetails;
+this.peerNodeDetails = peerNodeDetails;
+ }
+
+ public OMNodeDetails getLocalNodeDetails() {
+return localNodeDetails;
+ }
+
+ public List< OMNodeDetails > getPeerNodeDetails() {
+return peerNodeDetails;
+ }
+
+
+ /**
+ * Inspects and loads OM node configurations.
+ *
+ * If {@link OMConfigKeys#OZONE_OM_SERVICE_IDS_KEY} is configured with
+ * multiple ids and/ or if {@link OMConfigKeys#OZONE_OM_NODE_ID_KEY} is not
+ * specifically configured , this method determines the omServiceId
+ * and omNodeId by matching the node's address with the configured
+ * addresses. When a match is found, it sets the omServicId and omNodeId from
+ * the corresponding configuration key. This method also finds the OM peers
+ * nodes belonging to the same OM service.
+ *
+ * @param conf
+ */
+ public static OMHANodeDetails loadOMHAConfig(OzoneConfiguration conf) {
+InetSocketAddress localRpcAddress = null;
+String localOMServiceId = null;
+String localOMNodeId = null;
+int localRatisPort = 0;
+Collection omServiceIds = conf.getTrimmedStringCollection(
+OZONE_OM_SERVICE_IDS_KEY);
+
+String knownOMNodeId = conf.get(OZONE_OM_NODE_ID_KEY);
+int found = 0;
+boolean isOMAddressSet = false;
+
+for (String serviceId : omServiceIds) {
+ Collection omNodeIds = OmUtils.getOMNodeIds(conf, serviceId);
+
+ if (omNodeIds.size() == 0) {
+String msg = "Configuration does not have any value set for " +
+OZONE_OM_NODES_KEY + " for service ID " + serviceId + ". List of "
+
+"OM Node ID's should be specified for the service ID";
+throw new OzoneIllegalArgumentException(msg);
+ }
+
+ List peerNodesList = new ArrayList<>();
+ boolean isPeer;
+ for (String nodeId : omNodeIds) {
+if (knownOMNodeId != null && !knownOMNodeId.equals(nodeId)) {
+ isPeer = true;
+} else {
+
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=318558=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318558
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 25/Sep/19 20:05
Start Date: 25/Sep/19 20:05
Worklog Time Spent: 10m
Work Description: arp7 commented on pull request #1511: HDDS-2162. Make
Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328315080
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ha/OMHANodeDetails.java
##
@@ -0,0 +1,309 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.ha;
+
+import com.google.common.base.Preconditions;
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.OzoneConsts;
+import org.apache.hadoop.ozone.OzoneIllegalArgumentException;
+import org.apache.hadoop.ozone.om.OMConfigKeys;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_ADDRESS_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODES_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODE_ID_KEY;
+import static
org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_DEFAULT;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY;
+
+/**
+ * Class which maintains peer information and it's own OM node information.
+ */
+public class OMHANodeDetails {
+
+ public static final Logger LOG =
+ LoggerFactory.getLogger(OMHANodeDetails.class);
+ private final OMNodeDetails localNodeDetails;
+ private final List peerNodeDetails;
+
+ public OMHANodeDetails(OMNodeDetails localNodeDetails,
+ List peerNodeDetails) {
+this.localNodeDetails = localNodeDetails;
+this.peerNodeDetails = peerNodeDetails;
+ }
+
+ public OMNodeDetails getLocalNodeDetails() {
+return localNodeDetails;
+ }
+
+ public List< OMNodeDetails > getPeerNodeDetails() {
+return peerNodeDetails;
+ }
+
+
+ /**
+ * Inspects and loads OM node configurations.
+ *
+ * If {@link OMConfigKeys#OZONE_OM_SERVICE_IDS_KEY} is configured with
+ * multiple ids and/ or if {@link OMConfigKeys#OZONE_OM_NODE_ID_KEY} is not
+ * specifically configured , this method determines the omServiceId
+ * and omNodeId by matching the node's address with the configured
+ * addresses. When a match is found, it sets the omServicId and omNodeId from
+ * the corresponding configuration key. This method also finds the OM peers
+ * nodes belonging to the same OM service.
+ *
+ * @param conf
+ */
+ public static OMHANodeDetails loadOMHAConfig(OzoneConfiguration conf) {
+InetSocketAddress localRpcAddress = null;
+String localOMServiceId = null;
+String localOMNodeId = null;
+int localRatisPort = 0;
+Collection omServiceIds = conf.getTrimmedStringCollection(
+OZONE_OM_SERVICE_IDS_KEY);
+
+String knownOMNodeId = conf.get(OZONE_OM_NODE_ID_KEY);
+int found = 0;
+boolean isOMAddressSet = false;
+
+for (String serviceId : omServiceIds) {
+ Collection omNodeIds = OmUtils.getOMNodeIds(conf, serviceId);
+
+ if (omNodeIds.size() == 0) {
+String msg = "Configuration does not have any value set for " +
+OZONE_OM_NODES_KEY + " for service ID " + serviceId + ". List of "
+
+"OM Node ID's should be specified for the service ID";
+throw new OzoneIllegalArgumentException(msg);
+ }
+
+ List peerNodesList = new ArrayList<>();
+ boolean isPeer;
+ for (String nodeId : omNodeIds) {
+if (knownOMNodeId != null && !knownOMNodeId.equals(nodeId)) {
+ isPeer = true;
+} else {
+
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=318541=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318541
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 25/Sep/19 19:56
Start Date: 25/Sep/19 19:56
Worklog Time Spent: 10m
Work Description: arp7 commented on pull request #1511: HDDS-2162. Make
Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328311433
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ha/OMHANodeDetails.java
##
@@ -0,0 +1,309 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.ha;
+
+import com.google.common.base.Preconditions;
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.OzoneConsts;
+import org.apache.hadoop.ozone.OzoneIllegalArgumentException;
+import org.apache.hadoop.ozone.om.OMConfigKeys;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_ADDRESS_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODES_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODE_ID_KEY;
+import static
org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_DEFAULT;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY;
+
+/**
+ * Class which maintains peer information and it's own OM node information.
+ */
+public class OMHANodeDetails {
+
+ public static final Logger LOG =
+ LoggerFactory.getLogger(OMHANodeDetails.class);
+ private final OMNodeDetails localNodeDetails;
+ private final List peerNodeDetails;
+
+ public OMHANodeDetails(OMNodeDetails localNodeDetails,
+ List peerNodeDetails) {
+this.localNodeDetails = localNodeDetails;
+this.peerNodeDetails = peerNodeDetails;
+ }
+
+ public OMNodeDetails getLocalNodeDetails() {
+return localNodeDetails;
+ }
+
+ public List< OMNodeDetails > getPeerNodeDetails() {
+return peerNodeDetails;
+ }
+
+
+ /**
+ * Inspects and loads OM node configurations.
+ *
+ * If {@link OMConfigKeys#OZONE_OM_SERVICE_IDS_KEY} is configured with
+ * multiple ids and/ or if {@link OMConfigKeys#OZONE_OM_NODE_ID_KEY} is not
+ * specifically configured , this method determines the omServiceId
+ * and omNodeId by matching the node's address with the configured
+ * addresses. When a match is found, it sets the omServicId and omNodeId from
+ * the corresponding configuration key. This method also finds the OM peers
+ * nodes belonging to the same OM service.
+ *
+ * @param conf
+ */
+ public static OMHANodeDetails loadOMHAConfig(OzoneConfiguration conf) {
+InetSocketAddress localRpcAddress = null;
+String localOMServiceId = null;
+String localOMNodeId = null;
+int localRatisPort = 0;
+Collection omServiceIds = conf.getTrimmedStringCollection(
+OZONE_OM_SERVICE_IDS_KEY);
+
+String knownOMNodeId = conf.get(OZONE_OM_NODE_ID_KEY);
+int found = 0;
+boolean isOMAddressSet = false;
+
+for (String serviceId : omServiceIds) {
+ Collection omNodeIds = OmUtils.getOMNodeIds(conf, serviceId);
+
+ if (omNodeIds.size() == 0) {
+String msg = "Configuration does not have any value set for " +
+OZONE_OM_NODES_KEY + " for service ID " + serviceId + ". List of "
+
+"OM Node ID's should be specified for the service ID";
+throw new OzoneIllegalArgumentException(msg);
+ }
+
+ List peerNodesList = new ArrayList<>();
+ boolean isPeer;
+ for (String nodeId : omNodeIds) {
+if (knownOMNodeId != null && !knownOMNodeId.equals(nodeId)) {
+ isPeer = true;
+} else {
+
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=318533=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318533
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 25/Sep/19 19:51
Start Date: 25/Sep/19 19:51
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1511:
HDDS-2162. Make Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328309300
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ha/OMHANodeDetails.java
##
@@ -0,0 +1,309 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.ha;
+
+import com.google.common.base.Preconditions;
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.OzoneConsts;
+import org.apache.hadoop.ozone.OzoneIllegalArgumentException;
+import org.apache.hadoop.ozone.om.OMConfigKeys;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_ADDRESS_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODES_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODE_ID_KEY;
+import static
org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_DEFAULT;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY;
+
+/**
+ * Class which maintains peer information and it's own OM node information.
+ */
+public class OMHANodeDetails {
+
+ public static final Logger LOG =
+ LoggerFactory.getLogger(OMHANodeDetails.class);
+ private final OMNodeDetails localNodeDetails;
+ private final List peerNodeDetails;
+
+ public OMHANodeDetails(OMNodeDetails localNodeDetails,
+ List peerNodeDetails) {
+this.localNodeDetails = localNodeDetails;
+this.peerNodeDetails = peerNodeDetails;
+ }
+
+ public OMNodeDetails getLocalNodeDetails() {
+return localNodeDetails;
+ }
+
+ public List< OMNodeDetails > getPeerNodeDetails() {
+return peerNodeDetails;
+ }
+
+
+ /**
+ * Inspects and loads OM node configurations.
+ *
+ * If {@link OMConfigKeys#OZONE_OM_SERVICE_IDS_KEY} is configured with
+ * multiple ids and/ or if {@link OMConfigKeys#OZONE_OM_NODE_ID_KEY} is not
+ * specifically configured , this method determines the omServiceId
+ * and omNodeId by matching the node's address with the configured
+ * addresses. When a match is found, it sets the omServicId and omNodeId from
+ * the corresponding configuration key. This method also finds the OM peers
+ * nodes belonging to the same OM service.
+ *
+ * @param conf
+ */
+ public static OMHANodeDetails loadOMHAConfig(OzoneConfiguration conf) {
+InetSocketAddress localRpcAddress = null;
+String localOMServiceId = null;
+String localOMNodeId = null;
+int localRatisPort = 0;
+Collection omServiceIds = conf.getTrimmedStringCollection(
+OZONE_OM_SERVICE_IDS_KEY);
+
+String knownOMNodeId = conf.get(OZONE_OM_NODE_ID_KEY);
Review comment:
Isn't there also a key for our own service ID?
No, we don't have.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 318533)
Time Spent: 1h (was: 50m)
> Make Kerberos related configuration support HA style config
> ---
>
> Key: HDDS-2162
> URL:
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=318530=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318530
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 25/Sep/19 19:44
Start Date: 25/Sep/19 19:44
Worklog Time Spent: 10m
Work Description: arp7 commented on pull request #1511: HDDS-2162. Make
Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328306380
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ha/OMHANodeDetails.java
##
@@ -0,0 +1,309 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.ha;
+
+import com.google.common.base.Preconditions;
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.OzoneConsts;
+import org.apache.hadoop.ozone.OzoneIllegalArgumentException;
+import org.apache.hadoop.ozone.om.OMConfigKeys;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_ADDRESS_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODES_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODE_ID_KEY;
+import static
org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_DEFAULT;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY;
+
+/**
+ * Class which maintains peer information and it's own OM node information.
+ */
+public class OMHANodeDetails {
+
+ public static final Logger LOG =
+ LoggerFactory.getLogger(OMHANodeDetails.class);
+ private final OMNodeDetails localNodeDetails;
+ private final List peerNodeDetails;
+
+ public OMHANodeDetails(OMNodeDetails localNodeDetails,
+ List peerNodeDetails) {
+this.localNodeDetails = localNodeDetails;
+this.peerNodeDetails = peerNodeDetails;
+ }
+
+ public OMNodeDetails getLocalNodeDetails() {
+return localNodeDetails;
+ }
+
+ public List< OMNodeDetails > getPeerNodeDetails() {
+return peerNodeDetails;
+ }
+
+
+ /**
+ * Inspects and loads OM node configurations.
+ *
+ * If {@link OMConfigKeys#OZONE_OM_SERVICE_IDS_KEY} is configured with
+ * multiple ids and/ or if {@link OMConfigKeys#OZONE_OM_NODE_ID_KEY} is not
+ * specifically configured , this method determines the omServiceId
+ * and omNodeId by matching the node's address with the configured
+ * addresses. When a match is found, it sets the omServicId and omNodeId from
+ * the corresponding configuration key. This method also finds the OM peers
+ * nodes belonging to the same OM service.
+ *
+ * @param conf
+ */
+ public static OMHANodeDetails loadOMHAConfig(OzoneConfiguration conf) {
+InetSocketAddress localRpcAddress = null;
+String localOMServiceId = null;
+String localOMNodeId = null;
+int localRatisPort = 0;
+Collection omServiceIds = conf.getTrimmedStringCollection(
+OZONE_OM_SERVICE_IDS_KEY);
+
+String knownOMNodeId = conf.get(OZONE_OM_NODE_ID_KEY);
Review comment:
Also you should trim the string here after getting from conf.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 318530)
Time Spent: 50m (was: 40m)
> Make Kerberos related configuration support HA style config
> ---
>
> Key: HDDS-2162
> URL:
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[
https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=318529=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318529
]
ASF GitHub Bot logged work on HDDS-2162:
Author: ASF GitHub Bot
Created on: 25/Sep/19 19:42
Start Date: 25/Sep/19 19:42
Worklog Time Spent: 10m
Work Description: arp7 commented on pull request #1511: HDDS-2162. Make
Kerberos related configuration support HA style config.
URL: https://github.com/apache/hadoop/pull/1511#discussion_r328305846
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ha/OMHANodeDetails.java
##
@@ -0,0 +1,309 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.ha;
+
+import com.google.common.base.Preconditions;
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.OzoneConsts;
+import org.apache.hadoop.ozone.OzoneIllegalArgumentException;
+import org.apache.hadoop.ozone.om.OMConfigKeys;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.net.InetSocketAddress;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_ADDRESS_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODES_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_NODE_ID_KEY;
+import static
org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_DEFAULT;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_RATIS_PORT_KEY;
+import static org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_OM_SERVICE_IDS_KEY;
+
+/**
+ * Class which maintains peer information and it's own OM node information.
+ */
+public class OMHANodeDetails {
+
+ public static final Logger LOG =
+ LoggerFactory.getLogger(OMHANodeDetails.class);
+ private final OMNodeDetails localNodeDetails;
+ private final List peerNodeDetails;
+
+ public OMHANodeDetails(OMNodeDetails localNodeDetails,
+ List peerNodeDetails) {
+this.localNodeDetails = localNodeDetails;
+this.peerNodeDetails = peerNodeDetails;
+ }
+
+ public OMNodeDetails getLocalNodeDetails() {
+return localNodeDetails;
+ }
+
+ public List< OMNodeDetails > getPeerNodeDetails() {
+return peerNodeDetails;
+ }
+
+
+ /**
+ * Inspects and loads OM node configurations.
+ *
+ * If {@link OMConfigKeys#OZONE_OM_SERVICE_IDS_KEY} is configured with
+ * multiple ids and/ or if {@link OMConfigKeys#OZONE_OM_NODE_ID_KEY} is not
+ * specifically configured , this method determines the omServiceId
+ * and omNodeId by matching the node's address with the configured
+ * addresses. When a match is found, it sets the omServicId and omNodeId from
+ * the corresponding configuration key. This method also finds the OM peers
+ * nodes belonging to the same OM service.
+ *
+ * @param conf
+ */
+ public static OMHANodeDetails loadOMHAConfig(OzoneConfiguration conf) {
+InetSocketAddress localRpcAddress = null;
+String localOMServiceId = null;
+String localOMNodeId = null;
+int localRatisPort = 0;
+Collection omServiceIds = conf.getTrimmedStringCollection(
+OZONE_OM_SERVICE_IDS_KEY);
+
+String knownOMNodeId = conf.get(OZONE_OM_NODE_ID_KEY);
Review comment:
Isn't there also a key for our own service ID?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 318529)
Time Spent: 40m (was: 0.5h)
> Make Kerberos related configuration support HA style config
> ---
>
> Key: HDDS-2162
> URL: https://issues.apache.org/jira/browse/HDDS-2162
>
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[ https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=317853=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317853 ] ASF GitHub Bot logged work on HDDS-2162: Author: ASF GitHub Bot Created on: 24/Sep/19 21:12 Start Date: 24/Sep/19 21:12 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1511: HDDS-2162. Make Kerberos related configuration support HA style config. URL: https://github.com/apache/hadoop/pull/1511#issuecomment-534750574 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 79 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 32 | Maven dependency ordering for branch | | -1 | mvninstall | 33 | hadoop-hdds in trunk failed. | | -1 | mvninstall | 26 | hadoop-ozone in trunk failed. | | -1 | compile | 19 | hadoop-hdds in trunk failed. | | -1 | compile | 13 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 61 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 952 | branch has no errors when building and testing our client artifacts. | | -1 | javadoc | 19 | hadoop-hdds in trunk failed. | | -1 | javadoc | 18 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 1044 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 30 | hadoop-hdds in trunk failed. | | -1 | findbugs | 19 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | 0 | mvndep | 15 | Maven dependency ordering for patch | | -1 | mvninstall | 31 | hadoop-hdds in the patch failed. | | -1 | mvninstall | 28 | hadoop-ozone in the patch failed. | | -1 | compile | 20 | hadoop-hdds in the patch failed. | | -1 | compile | 15 | hadoop-ozone in the patch failed. | | -1 | javac | 20 | hadoop-hdds in the patch failed. | | -1 | javac | 15 | hadoop-ozone in the patch failed. | | +1 | checkstyle | 54 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 786 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 18 | hadoop-hdds in the patch failed. | | -1 | javadoc | 17 | hadoop-ozone in the patch failed. | | -1 | findbugs | 27 | hadoop-hdds in the patch failed. | | -1 | findbugs | 17 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | -1 | unit | 25 | hadoop-hdds in the patch failed. | | -1 | unit | 19 | hadoop-ozone in the patch failed. | | +1 | asflicense | 29 | The patch does not generate ASF License warnings. | | | | 2514 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1511 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux cff901010f89 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 66400c1 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/2/artifact/out/branch-mvninstall-hadoop-hdds.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/2/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/2/artifact/out/branch-compile-hadoop-hdds.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/2/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/2/artifact/out/branch-javadoc-hadoop-hdds.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/2/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/2/artifact/out/branch-findbugs-hadoop-hdds.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/2/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/2/artifact/out/patch-mvninstall-hadoop-hdds.txt | | mvninstall |
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[ https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=317107=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317107 ] ASF GitHub Bot logged work on HDDS-2162: Author: ASF GitHub Bot Created on: 24/Sep/19 01:31 Start Date: 24/Sep/19 01:31 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1511: HDDS-2162. Make Kerberos related configuration support HA style config. URL: https://github.com/apache/hadoop/pull/1511#issuecomment-534348793 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 44 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 23 | Maven dependency ordering for branch | | -1 | mvninstall | 32 | hadoop-hdds in trunk failed. | | -1 | mvninstall | 26 | hadoop-ozone in trunk failed. | | -1 | compile | 21 | hadoop-hdds in trunk failed. | | -1 | compile | 15 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 59 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 870 | branch has no errors when building and testing our client artifacts. | | -1 | javadoc | 22 | hadoop-hdds in trunk failed. | | -1 | javadoc | 20 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 968 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 30 | hadoop-hdds in trunk failed. | | -1 | findbugs | 21 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | 0 | mvndep | 18 | Maven dependency ordering for patch | | -1 | mvninstall | 35 | hadoop-hdds in the patch failed. | | -1 | mvninstall | 29 | hadoop-ozone in the patch failed. | | -1 | compile | 26 | hadoop-hdds in the patch failed. | | -1 | compile | 19 | hadoop-ozone in the patch failed. | | -1 | javac | 26 | hadoop-hdds in the patch failed. | | -1 | javac | 19 | hadoop-ozone in the patch failed. | | -0 | checkstyle | 30 | hadoop-ozone: The patch generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 734 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 22 | hadoop-hdds in the patch failed. | | -1 | javadoc | 20 | hadoop-ozone in the patch failed. | | -1 | findbugs | 31 | hadoop-hdds in the patch failed. | | -1 | findbugs | 20 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | -1 | unit | 29 | hadoop-hdds in the patch failed. | | -1 | unit | 23 | hadoop-ozone in the patch failed. | | +1 | asflicense | 32 | The patch does not generate ASF License warnings. | | | | 2420 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1511 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 0f79de22ddec 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / e8e7d7b | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/1/artifact/out/branch-mvninstall-hadoop-hdds.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/1/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/1/artifact/out/branch-compile-hadoop-hdds.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/1/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/1/artifact/out/branch-javadoc-hadoop-hdds.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/1/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/1/artifact/out/branch-findbugs-hadoop-hdds.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/1/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1511/1/artifact/out/patch-mvninstall-hadoop-hdds.txt | | mvninstall |
[jira] [Work logged] (HDDS-2162) Make Kerberos related configuration support HA style config
[ https://issues.apache.org/jira/browse/HDDS-2162?focusedWorklogId=317100=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317100 ] ASF GitHub Bot logged work on HDDS-2162: Author: ASF GitHub Bot Created on: 24/Sep/19 00:49 Start Date: 24/Sep/19 00:49 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on pull request #1511: HDDS-2162. Make Kerberos related configuration support HA style config. URL: https://github.com/apache/hadoop/pull/1511 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 317100) Remaining Estimate: 0h Time Spent: 10m > Make Kerberos related configuration support HA style config > --- > > Key: HDDS-2162 > URL: https://issues.apache.org/jira/browse/HDDS-2162 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > To have a single configuration to use across OM cluster, few of the configs > like > OZONE_OM_KERBEROS_KEYTAB_FILE_KEY, > OZONE_OM_KERBEROS_PRINCIPAL_KEY, > OZONE_OM_HTTP_KERBEROS_KEYTAB_FILE, > OZONE_OM_HTTP_KERBEROS_PRINCIPAL_KEY need to support configs which append > with service id and node id. > > This Jira is to fix the above configs. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
