[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3539: - Fix Version/s: 2.8.0 > Compatibility doc to state that ATS v1 is a stable REST API > --- > > Key: YARN-3539 > URL: https://issues.apache.org/jira/browse/YARN-3539 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Affects Versions: 2.7.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: 2.8.0, 2.7.1, 3.0.0-alpha1 > > Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, > TimelineServer.html, YARN-3539-003.patch, YARN-3539-004.patch, > YARN-3539-005.patch, YARN-3539-006.patch, YARN-3539-007.patch, > YARN-3539-008.patch, YARN-3539-009.patch, YARN-3539-010.patch, > YARN-3539.11.patch, timeline_get_api_examples.txt > > > The ATS v2 discussion and YARN-2423 have raised the question: "how stable are > the ATSv1 APIs"? > The existing compatibility document actually states that the History Server > is [a stable REST > API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], > which effectively means that ATSv1 has already been declared as a stable API. > Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3539: -- Attachment: YARN-3539.11.patch The last patch looks good to me, thought there're two nits in it. I fixed them directly in the patch. Will commit it once jenkins comment again. > Compatibility doc to state that ATS v1 is a stable REST API > --- > > Key: YARN-3539 > URL: https://issues.apache.org/jira/browse/YARN-3539 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Affects Versions: 2.7.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, > TimelineServer.html, YARN-3539-003.patch, YARN-3539-004.patch, > YARN-3539-005.patch, YARN-3539-006.patch, YARN-3539-007.patch, > YARN-3539-008.patch, YARN-3539-009.patch, YARN-3539-010.patch, > YARN-3539.11.patch, timeline_get_api_examples.txt > > > The ATS v2 discussion and YARN-2423 have raised the question: "how stable are > the ATSv1 APIs"? > The existing compatibility document actually states that the History Server > is [a stable REST > API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], > which effectively means that ATSv1 has already been declared as a stable API. > Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3539: -- Labels: (was: BB2015-05-TBR) > Compatibility doc to state that ATS v1 is a stable REST API > --- > > Key: YARN-3539 > URL: https://issues.apache.org/jira/browse/YARN-3539 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Affects Versions: 2.7.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, > TimelineServer.html, YARN-3539-003.patch, YARN-3539-004.patch, > YARN-3539-005.patch, YARN-3539-006.patch, YARN-3539-007.patch, > YARN-3539-008.patch, YARN-3539-009.patch, YARN-3539-010.patch, > timeline_get_api_examples.txt > > > The ATS v2 discussion and YARN-2423 have raised the question: "how stable are > the ATSv1 APIs"? > The existing compatibility document actually states that the History Server > is [a stable REST > API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], > which effectively means that ATSv1 has already been declared as a stable API. > Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated YARN-3539: - Attachment: YARN-3539-010.patch Patch -010. in sync with trunk and the now-deleted {{index.md}}; declares the REST API in the {{site.xml}} indexes instead. > Compatibility doc to state that ATS v1 is a stable REST API > --- > > Key: YARN-3539 > URL: https://issues.apache.org/jira/browse/YARN-3539 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Affects Versions: 2.7.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Labels: BB2015-05-TBR > Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, > TimelineServer.html, YARN-3539-003.patch, YARN-3539-004.patch, > YARN-3539-005.patch, YARN-3539-006.patch, YARN-3539-007.patch, > YARN-3539-008.patch, YARN-3539-009.patch, YARN-3539-010.patch, > timeline_get_api_examples.txt > > > The ATS v2 discussion and YARN-2423 have raised the question: "how stable are > the ATSv1 APIs"? > The existing compatibility document actually states that the History Server > is [a stable REST > API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], > which effectively means that ATSv1 has already been declared as a stable API. > Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated YARN-3539: - Attachment: YARN-3539-009.patch sorry, missed that last update this patch fixes the table formatting and adds the {{originalUrl}} field that is in the JSON examples but not in the table. Interesting, in slider our [SerializedApplicationReport|https://github.com/apache/incubator-slider/blob/b70d830aee6fc0171cb36fff0604b310dc565e3e/slider-core/src/main/java/org/apache/slider/core/launch/SerializedApplicationReport.java] data type is almost identical; If I'd known of this one I'd have skipped re-implementing it. not fixing whitespace as per the comment in the review, {{--whitespace=fix}} does that when the patch is actually merged in. Instead I'll see if we can get that part of the patch review downgraded from a -1 to a warning. > Compatibility doc to state that ATS v1 is a stable REST API > --- > > Key: YARN-3539 > URL: https://issues.apache.org/jira/browse/YARN-3539 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Affects Versions: 2.7.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Labels: BB2015-05-TBR > Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, > TimelineServer.html, YARN-3539-003.patch, YARN-3539-004.patch, > YARN-3539-005.patch, YARN-3539-006.patch, YARN-3539-007.patch, > YARN-3539-008.patch, YARN-3539-009.patch, timeline_get_api_examples.txt > > > The ATS v2 discussion and YARN-2423 have raised the question: "how stable are > the ATSv1 APIs"? > The existing compatibility document actually states that the History Server > is [a stable REST > API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], > which effectively means that ATSv1 has already been declared as a stable API. > Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3539: -- Attachment: TimelineServer.html > Compatibility doc to state that ATS v1 is a stable REST API > --- > > Key: YARN-3539 > URL: https://issues.apache.org/jira/browse/YARN-3539 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Affects Versions: 2.7.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Labels: BB2015-05-TBR > Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, > TimelineServer.html, YARN-3539-003.patch, YARN-3539-004.patch, > YARN-3539-005.patch, YARN-3539-006.patch, YARN-3539-007.patch, > YARN-3539-008.patch, timeline_get_api_examples.txt > > > The ATS v2 discussion and YARN-2423 have raised the question: "how stable are > the ATSv1 APIs"? > The existing compatibility document actually states that the History Server > is [a stable REST > API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], > which effectively means that ATSv1 has already been declared as a stable API. > Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated YARN-3539: - Attachment: YARN-3539-008.patch Patch with comments and corrections applied. Also * cut down the lines that were one paragraph/line long * reviewed and edited opening section. * marked up all default values as `code` in the tables of YARN configuration options. > Compatibility doc to state that ATS v1 is a stable REST API > --- > > Key: YARN-3539 > URL: https://issues.apache.org/jira/browse/YARN-3539 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Affects Versions: 2.7.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Labels: BB2015-05-TBR > Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, > YARN-3539-003.patch, YARN-3539-004.patch, YARN-3539-005.patch, > YARN-3539-006.patch, YARN-3539-007.patch, YARN-3539-008.patch, > timeline_get_api_examples.txt > > > The ATS v2 discussion and YARN-2423 have raised the question: "how stable are > the ATSv1 APIs"? > The existing compatibility document actually states that the History Server > is [a stable REST > API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], > which effectively means that ATSv1 has already been declared as a stable API. > Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated YARN-3539: - Attachment: YARN-3539-007.patch Patch 007 # adds examples from timeline doc # ..but omits the YARN records, that should be documented standalone # tags org/apache/hadoop/yarn/server/webapp/dao as public and evolving. I've left all the protobuf API& records alone, as that is a separate issue. This JIRA only covers the REST API. > Compatibility doc to state that ATS v1 is a stable REST API > --- > > Key: YARN-3539 > URL: https://issues.apache.org/jira/browse/YARN-3539 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Affects Versions: 2.7.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Labels: BB2015-05-TBR > Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, > YARN-3539-003.patch, YARN-3539-004.patch, YARN-3539-005.patch, > YARN-3539-006.patch, YARN-3539-007.patch, timeline_get_api_examples.txt > > > The ATS v2 discussion and YARN-2423 have raised the question: "how stable are > the ATSv1 APIs"? > The existing compatibility document actually states that the History Server > is [a stable REST > API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], > which effectively means that ATSv1 has already been declared as a stable API. > Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated YARN-3539: --- Labels: BB2015-05-TBR (was: ) > Compatibility doc to state that ATS v1 is a stable REST API > --- > > Key: YARN-3539 > URL: https://issues.apache.org/jira/browse/YARN-3539 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Affects Versions: 2.7.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Labels: BB2015-05-TBR > Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, > YARN-3539-003.patch, YARN-3539-004.patch, YARN-3539-005.patch, > YARN-3539-006.patch, timeline_get_api_examples.txt > > > The ATS v2 discussion and YARN-2423 have raised the question: "how stable are > the ATSv1 APIs"? > The existing compatibility document actually states that the History Server > is [a stable REST > API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], > which effectively means that ATSv1 has already been declared as a stable API. > Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3539: -- Attachment: timeline_get_api_examples.txt I attached a document of the timeline APIs' output examples. We can make use of it for the documentation. > Compatibility doc to state that ATS v1 is a stable REST API > --- > > Key: YARN-3539 > URL: https://issues.apache.org/jira/browse/YARN-3539 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Affects Versions: 2.7.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, > YARN-3539-003.patch, YARN-3539-004.patch, YARN-3539-005.patch, > YARN-3539-006.patch, timeline_get_api_examples.txt > > > The ATS v2 discussion and YARN-2423 have raised the question: "how stable are > the ATSv1 APIs"? > The existing compatibility document actually states that the History Server > is [a stable REST > API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], > which effectively means that ATSv1 has already been declared as a stable API. > Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated YARN-3539: - Attachment: YARN-3539-006.patch Patch -006 also uprates all of the data structures and {{TimelineClient}} from {{@Unstable}} to {{@Evolving}} > Compatibility doc to state that ATS v1 is a stable REST API > --- > > Key: YARN-3539 > URL: https://issues.apache.org/jira/browse/YARN-3539 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Affects Versions: 2.7.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, > YARN-3539-003.patch, YARN-3539-004.patch, YARN-3539-005.patch, > YARN-3539-006.patch > > > The ATS v2 discussion and YARN-2423 have raised the question: "how stable are > the ATSv1 APIs"? > The existing compatibility document actually states that the History Server > is [a stable REST > API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], > which effectively means that ATSv1 has already been declared as a stable API. > Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated YARN-3539: - Attachment: YARN-3539-005.patch patch -005 h3. Valid entity types. Currently "/" and " " are allowed in entity types. I'm not sure that is a good idea. I'd also worry about "<" and ">" for security reasons. A regexp of A-Za-z0-9 + others of a limited set may be enough. We can always check with all known users to see what entity types they are using. Locking it down for V1 allows v2 to be consistent. bq. 1. For the bullet points of "Current Status and Future Plans", can we organize them a bit better. For example, we partition them into the groups of a) current status and b) future plans. For bullet 4, not just history, but all timeline data. done bq. 2. Can we move "Timeline Server REST API" section before "Generic Data REST APIs"? done bq. 3. Application elements table seems to be wrongly formatted. I think that's why site compilation is failed. fixed bq. 4. "Generic Data REST APIs" output examples need to be slightly updated. Some more fields are added or changed. Those are the examples from YARN-1876. If you have some more up to date ones I'll replace them. done bq. 5. "Timeline Server REST API" output examples are not genuine. Perhaps, we can run a simple MR example job, and get the up-to-date timeline entity and application info to show as the examples. +1. Do you have this? > Compatibility doc to state that ATS v1 is a stable REST API > --- > > Key: YARN-3539 > URL: https://issues.apache.org/jira/browse/YARN-3539 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Affects Versions: 2.7.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, > YARN-3539-003.patch, YARN-3539-004.patch, YARN-3539-005.patch > > > The ATS v2 discussion and YARN-2423 have raised the question: "how stable are > the ATSv1 APIs"? > The existing compatibility document actually states that the History Server > is [a stable REST > API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], > which effectively means that ATSv1 has already been declared as a stable API. > Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated YARN-3539: - Attachment: YARN-3539-004.patch Patch -004 this merges in YARN-1876, converting that to Markdown and replacing anything in patch -003 which is covered in YARN-1876. The only content retained from the 003 patch is # error codes # domains # a reference (but no detail) on the response to a timeline entity POST. Because this patch now contains YARN-1876, with its examples, it is a lot more thorough. One area that is not covered in the ATSv1 API is *what constitutes a valid entity type or domain?*. This is important, as the API & documentation implies that "a/b" is a valid ID...it may be accepted as such, but to do so would be an error. There is also the fact that the {{/domain}} path was added under {{/ws/v1/timeline/}}, so matches the path of entity types. Can you have an entity type called "domain"? Was it previously possible? I would propose # strictly defining what constitutes a valid entity type via a regular expression, and declaring whether the types are case sensitive. # declaring whether {{domain}} is a valid entity type Apart from {{/domain}}, there doesn't appear to be any restriction on entity type names. Any new path added under the {{/ws/v1/timeline/}} is therefore at risk of clashing with an applications use of ATS. Even a simple {{about/}} path is potentially backwards-incompatible, so cannot be added if the API is to be declared stable. Either a new path is used for extensions, or a once-and-for-all subpath is defined for extra attributes, e.g. {{/ws/v1/timeline/state}}, and that subpath (here {{state}}) declared to be an invalid entity type. > Compatibility doc to state that ATS v1 is a stable REST API > --- > > Key: YARN-3539 > URL: https://issues.apache.org/jira/browse/YARN-3539 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Affects Versions: 2.7.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, > YARN-3539-003.patch, YARN-3539-004.patch > > > The ATS v2 discussion and YARN-2423 have raised the question: "how stable are > the ATSv1 APIs"? > The existing compatibility document actually states that the History Server > is [a stable REST > API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], > which effectively means that ATSv1 has already been declared as a stable API. > Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated YARN-3539: - Attachment: YARN-3539-003.patch Patch -003 updates {{timelineserver.md}} # specify the REST API (non-normative) # add some more on futures # review configuration options # fix up broken internal links by adding the anchors # yarn/index.html includes link to ATS Rest API as one of the listed REST APIs > Compatibility doc to state that ATS v1 is a stable REST API > --- > > Key: YARN-3539 > URL: https://issues.apache.org/jira/browse/YARN-3539 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Affects Versions: 2.7.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, > YARN-3539-003.patch > > > The ATS v2 discussion and YARN-2423 have raised the question: "how stable are > the ATSv1 APIs"? > The existing compatibility document actually states that the History Server > is [a stable REST > API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], > which effectively means that ATSv1 has already been declared as a stable API. > Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)