Gour Saha created YARN-2678: ------------------------------- Summary: Recommended improvements to Yarn Registry Key: YARN-2678 URL: https://issues.apache.org/jira/browse/YARN-2678 Project: Hadoop YARN Issue Type: Sub-task Reporter: Gour Saha
In the process of binding to Slider AM from Slider agent python code here are some of the items I stumbled upon and would recommend as improvements. This is how the Slider's registry looks today - {noformat} jsonservicerec{ "description" : "Slider Application Master", "external" : [ { "api" : "org.apache.slider.appmaster", "addressType" : "host/port", "protocolType" : "hadoop/protobuf", "addresses" : [ [ "c6408.ambari.apache.org", "34837" ] ] }, { "api" : "org.apache.http.UI", "addressType" : "uri", "protocolType" : "webui", "addresses" : [ [ "http://c6408.ambari.apache.org:43314" ] ] }, { "api" : "org.apache.slider.management", "addressType" : "uri", "protocolType" : "REST", "addresses" : [ [ "http://c6408.ambari.apache.org:43314/ws/v1/slider/mgmt" ] ] }, { "api" : "org.apache.slider.publisher", "addressType" : "uri", "protocolType" : "REST", "addresses" : [ [ "http://c6408.ambari.apache.org:43314/ws/v1/slider/publisher" ] ] }, { "api" : "org.apache.slider.registry", "addressType" : "uri", "protocolType" : "REST", "addresses" : [ [ "http://c6408.ambari.apache.org:43314/ws/v1/slider/registry" ] ] }, { "api" : "org.apache.slider.publisher.configurations", "addressType" : "uri", "protocolType" : "REST", "addresses" : [ [ "http://c6408.ambari.apache.org:43314/ws/v1/slider/publisher/slider" ] ] } ], "internal" : [ { "api" : "org.apache.slider.agents.secure", "addressType" : "uri", "protocolType" : "REST", "addresses" : [ [ "https://c6408.ambari.apache.org:46958/ws/v1/slider/agents" ] ] }, { "api" : "org.apache.slider.agents.oneway", "addressType" : "uri", "protocolType" : "REST", "addresses" : [ [ "https://c6408.ambari.apache.org:57513/ws/v1/slider/agents" ] ] } ], "yarn:persistence" : "application", "yarn:id" : "application_1412974695267_0015" } {noformat} Recommendations: 1. I would suggest to either remove the string {color:red}jsonservicerec{color} or if it is desirable to have a non-null data at all times then loop the string into the json structure as a top-level attribute to ensure that the registry data is always a valid json document. 2. The {color:red}addresses{color} attribute is currently a list of list. I would recommend to convert it to a list of dictionary objects. In the dictionary object it would be nice to have the host and port portions of objects of addressType "uri" as separate key-value pairs to avoid parsing on the client side. The URI should also be retained as a key say "uri" to avoid clients trying to generate it by concatenating host, port, resource-path, etc. Here is a proposed structure - {noformat} { ... "internal" : [ { "api" : "org.apache.slider.agents.secure", "addressType" : "uri", "protocolType" : "REST", "addresses" : [ { "uri" : "https://c6408.ambari.apache.org:46958/ws/v1/slider/agents", "host" : "c6408.ambari.apache.org", "port": "46958" } ] } ], } {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)