The REST APIs are covered under the compatibility guidelines[1]. Presuming these are under a new API version number, it's not clear to me from the existing guidelines if adding one is okay in a maintenance release. It sounds surprising to me.
[1]: https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs On Wed, Nov 15, 2017 at 9:23 PM, Daniel Templeton <[email protected]> wrote: > Looks like our REST endpoints return malformed JSON for any DAO that > includes a Map. That includes: > > * the resourceSecondsMap and preemptedResourceSecondsMap entries in all > the GET /apps/* endpoints, > * the operationsInfo entry in the GET /scheduler endpoint for capacity > scheduler, > * the local_resources, environment, and acls entries in the POST /apps > endpoint, and > * the labelsToNodes entry in the GET /label-mappings endpoint. > > The issue is that each entry in the map is included with a duplicate key > ("entry"). Some JSON parsers will choke on the error, and some will > quietly drop the duplicates. I've filed YARN-7505 to address the issue. > > The solution is to replace the Jersey JSON serializer with the Jackson > JSON serializer. This change fixes the issue, but it changes the structure > of the resulting JSON. For example, without YARN-7505, hitting /apps might > yield JSON that contains something like: > > "resourceSecondsMap":{ > "entry":{"key":"memory-mb","value":"11225"}, > "entry":{"key":"vcores","value":"5"} > "entry":{"key":"test","value":"0"} > "entry":{"key":"test2","value":"0"} > } > > With YARN-7505, we get: > > "resourceSecondsMap": { > "test2":0, > "test":0, > "memory-mb":11225, > "vcores":5 > } > > The first example is obviously broken, so the second one is clearly > better, but it's structurally different. > > For the GET /label-mappings endpoint, the keys of the map also have to be > changed to simple strings because JSON doesn't allow for complex map keys. > So this: > > "labelsToNodes":{ > "entry":{ > "key":{"name":"label1","exclusivity":"true"}, > "value":{"nodes":"localhost:63261"} > } > } > > becomes this: > > "labelsToNodes":{ > "label1":{ > "nodes":["dhcp-10-16-0-181.pa.cloudera.com:63261"] > } > } > > The first one sucks and is invalid, but changing to the second one will > break clients that are parsing the first one, especially if they're > expecting to get the label exclusivity from this endpoint. > > Before I try to get YARN-7505 committed, I want to give the community a > chance to voice any concerns about the change. It's too late to get into > 3.0.0, so we'd be looking at 3.0.1 and 3.1.0. > > Feel free to comment here or on the JIRA directly. > > Thanks, > Daniel > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > -- busbey
