[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3539: - Fix Version/s: 2.8.0 > Compatibility doc to state that ATS v1 is a stable REST API > --- > > Key: YARN-3539 > URL: https://issues.apache.org/jira/browse/YARN-3539 > Project: Hadoop YARN > Issue Type: Improvement > Components: documentation >Affects Versions: 2.7.0 >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: 2.8.0, 2.7.1, 3.0.0-alpha1 > > Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, > TimelineServer.html, YARN-3539-003.patch, YARN-3539-004.patch, > YARN-3539-005.patch, YARN-3539-006.patch, YARN-3539-007.patch, > YARN-3539-008.patch, YARN-3539-009.patch, YARN-3539-010.patch, > YARN-3539.11.patch, timeline_get_api_examples.txt > > > The ATS v2 discussion and YARN-2423 have raised the question: "how stable are > the ATSv1 APIs"? > The existing compatibility document actually states that the History Server > is [a stable REST > API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], > which effectively means that ATSv1 has already been declared as a stable API. > Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3539: -- Attachment: YARN-3539.11.patch The last patch looks good to me, thought there're two nits in it. I fixed them directly in the patch. Will commit it once jenkins comment again. Compatibility doc to state that ATS v1 is a stable REST API --- Key: YARN-3539 URL: https://issues.apache.org/jira/browse/YARN-3539 Project: Hadoop YARN Issue Type: Improvement Components: documentation Affects Versions: 2.7.0 Reporter: Steve Loughran Assignee: Steve Loughran Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, TimelineServer.html, YARN-3539-003.patch, YARN-3539-004.patch, YARN-3539-005.patch, YARN-3539-006.patch, YARN-3539-007.patch, YARN-3539-008.patch, YARN-3539-009.patch, YARN-3539-010.patch, YARN-3539.11.patch, timeline_get_api_examples.txt The ATS v2 discussion and YARN-2423 have raised the question: how stable are the ATSv1 APIs? The existing compatibility document actually states that the History Server is [a stable REST API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], which effectively means that ATSv1 has already been declared as a stable API. Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3539: -- Labels: (was: BB2015-05-TBR) Compatibility doc to state that ATS v1 is a stable REST API --- Key: YARN-3539 URL: https://issues.apache.org/jira/browse/YARN-3539 Project: Hadoop YARN Issue Type: Improvement Components: documentation Affects Versions: 2.7.0 Reporter: Steve Loughran Assignee: Steve Loughran Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, TimelineServer.html, YARN-3539-003.patch, YARN-3539-004.patch, YARN-3539-005.patch, YARN-3539-006.patch, YARN-3539-007.patch, YARN-3539-008.patch, YARN-3539-009.patch, YARN-3539-010.patch, timeline_get_api_examples.txt The ATS v2 discussion and YARN-2423 have raised the question: how stable are the ATSv1 APIs? The existing compatibility document actually states that the History Server is [a stable REST API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], which effectively means that ATSv1 has already been declared as a stable API. Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[
https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated YARN-3539:
-
Attachment: YARN-3539-009.patch
sorry, missed that last update
this patch fixes the table formatting and adds the {{originalUrl}} field that
is in the JSON examples but not in the table. Interesting, in slider our
[SerializedApplicationReport|https://github.com/apache/incubator-slider/blob/b70d830aee6fc0171cb36fff0604b310dc565e3e/slider-core/src/main/java/org/apache/slider/core/launch/SerializedApplicationReport.java]
data type is almost identical; If I'd known of this one I'd have skipped
re-implementing it.
not fixing whitespace as per the comment in the review, {{--whitespace=fix}}
does that when the patch is actually merged in. Instead I'll see if we can get
that part of the patch review downgraded from a -1 to a warning.
Compatibility doc to state that ATS v1 is a stable REST API
---
Key: YARN-3539
URL: https://issues.apache.org/jira/browse/YARN-3539
Project: Hadoop YARN
Issue Type: Improvement
Components: documentation
Affects Versions: 2.7.0
Reporter: Steve Loughran
Assignee: Steve Loughran
Labels: BB2015-05-TBR
Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch,
TimelineServer.html, YARN-3539-003.patch, YARN-3539-004.patch,
YARN-3539-005.patch, YARN-3539-006.patch, YARN-3539-007.patch,
YARN-3539-008.patch, YARN-3539-009.patch, timeline_get_api_examples.txt
The ATS v2 discussion and YARN-2423 have raised the question: how stable are
the ATSv1 APIs?
The existing compatibility document actually states that the History Server
is [a stable REST
API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs],
which effectively means that ATSv1 has already been declared as a stable API.
Clarify this by patching the compatibility document appropriately
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[
https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated YARN-3539:
-
Attachment: YARN-3539-010.patch
Patch -010. in sync with trunk and the now-deleted {{index.md}}; declares the
REST API in the {{site.xml}} indexes instead.
Compatibility doc to state that ATS v1 is a stable REST API
---
Key: YARN-3539
URL: https://issues.apache.org/jira/browse/YARN-3539
Project: Hadoop YARN
Issue Type: Improvement
Components: documentation
Affects Versions: 2.7.0
Reporter: Steve Loughran
Assignee: Steve Loughran
Labels: BB2015-05-TBR
Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch,
TimelineServer.html, YARN-3539-003.patch, YARN-3539-004.patch,
YARN-3539-005.patch, YARN-3539-006.patch, YARN-3539-007.patch,
YARN-3539-008.patch, YARN-3539-009.patch, YARN-3539-010.patch,
timeline_get_api_examples.txt
The ATS v2 discussion and YARN-2423 have raised the question: how stable are
the ATSv1 APIs?
The existing compatibility document actually states that the History Server
is [a stable REST
API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs],
which effectively means that ATSv1 has already been declared as a stable API.
Clarify this by patching the compatibility document appropriately
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3539: -- Attachment: TimelineServer.html Compatibility doc to state that ATS v1 is a stable REST API --- Key: YARN-3539 URL: https://issues.apache.org/jira/browse/YARN-3539 Project: Hadoop YARN Issue Type: Improvement Components: documentation Affects Versions: 2.7.0 Reporter: Steve Loughran Assignee: Steve Loughran Labels: BB2015-05-TBR Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, TimelineServer.html, YARN-3539-003.patch, YARN-3539-004.patch, YARN-3539-005.patch, YARN-3539-006.patch, YARN-3539-007.patch, YARN-3539-008.patch, timeline_get_api_examples.txt The ATS v2 discussion and YARN-2423 have raised the question: how stable are the ATSv1 APIs? The existing compatibility document actually states that the History Server is [a stable REST API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], which effectively means that ATSv1 has already been declared as a stable API. Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated YARN-3539: - Attachment: YARN-3539-008.patch Patch with comments and corrections applied. Also * cut down the lines that were one paragraph/line long * reviewed and edited opening section. * marked up all default values as `code` in the tables of YARN configuration options. Compatibility doc to state that ATS v1 is a stable REST API --- Key: YARN-3539 URL: https://issues.apache.org/jira/browse/YARN-3539 Project: Hadoop YARN Issue Type: Improvement Components: documentation Affects Versions: 2.7.0 Reporter: Steve Loughran Assignee: Steve Loughran Labels: BB2015-05-TBR Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, YARN-3539-003.patch, YARN-3539-004.patch, YARN-3539-005.patch, YARN-3539-006.patch, YARN-3539-007.patch, YARN-3539-008.patch, timeline_get_api_examples.txt The ATS v2 discussion and YARN-2423 have raised the question: how stable are the ATSv1 APIs? The existing compatibility document actually states that the History Server is [a stable REST API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], which effectively means that ATSv1 has already been declared as a stable API. Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated YARN-3539: - Attachment: YARN-3539-007.patch Patch 007 # adds examples from timeline doc # ..but omits the YARN records, that should be documented standalone # tags org/apache/hadoop/yarn/server/webapp/dao as public and evolving. I've left all the protobuf API records alone, as that is a separate issue. This JIRA only covers the REST API. Compatibility doc to state that ATS v1 is a stable REST API --- Key: YARN-3539 URL: https://issues.apache.org/jira/browse/YARN-3539 Project: Hadoop YARN Issue Type: Improvement Components: documentation Affects Versions: 2.7.0 Reporter: Steve Loughran Assignee: Steve Loughran Labels: BB2015-05-TBR Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, YARN-3539-003.patch, YARN-3539-004.patch, YARN-3539-005.patch, YARN-3539-006.patch, YARN-3539-007.patch, timeline_get_api_examples.txt The ATS v2 discussion and YARN-2423 have raised the question: how stable are the ATSv1 APIs? The existing compatibility document actually states that the History Server is [a stable REST API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], which effectively means that ATSv1 has already been declared as a stable API. Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated YARN-3539: --- Labels: BB2015-05-TBR (was: ) Compatibility doc to state that ATS v1 is a stable REST API --- Key: YARN-3539 URL: https://issues.apache.org/jira/browse/YARN-3539 Project: Hadoop YARN Issue Type: Improvement Components: documentation Affects Versions: 2.7.0 Reporter: Steve Loughran Assignee: Steve Loughran Labels: BB2015-05-TBR Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, YARN-3539-003.patch, YARN-3539-004.patch, YARN-3539-005.patch, YARN-3539-006.patch, timeline_get_api_examples.txt The ATS v2 discussion and YARN-2423 have raised the question: how stable are the ATSv1 APIs? The existing compatibility document actually states that the History Server is [a stable REST API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], which effectively means that ATSv1 has already been declared as a stable API. Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated YARN-3539: - Attachment: YARN-3539-005.patch patch -005 h3. Valid entity types. Currently / and are allowed in entity types. I'm not sure that is a good idea. I'd also worry about and for security reasons. A regexp of A-Za-z0-9 + others of a limited set may be enough. We can always check with all known users to see what entity types they are using. Locking it down for V1 allows v2 to be consistent. bq. 1. For the bullet points of Current Status and Future Plans, can we organize them a bit better. For example, we partition them into the groups of a) current status and b) future plans. For bullet 4, not just history, but all timeline data. done bq. 2. Can we move Timeline Server REST API section before Generic Data REST APIs? done bq. 3. Application elements table seems to be wrongly formatted. I think that's why site compilation is failed. fixed bq. 4. Generic Data REST APIs output examples need to be slightly updated. Some more fields are added or changed. Those are the examples from YARN-1876. If you have some more up to date ones I'll replace them. done bq. 5. Timeline Server REST API output examples are not genuine. Perhaps, we can run a simple MR example job, and get the up-to-date timeline entity and application info to show as the examples. +1. Do you have this? Compatibility doc to state that ATS v1 is a stable REST API --- Key: YARN-3539 URL: https://issues.apache.org/jira/browse/YARN-3539 Project: Hadoop YARN Issue Type: Improvement Components: documentation Affects Versions: 2.7.0 Reporter: Steve Loughran Assignee: Steve Loughran Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, YARN-3539-003.patch, YARN-3539-004.patch, YARN-3539-005.patch The ATS v2 discussion and YARN-2423 have raised the question: how stable are the ATSv1 APIs? The existing compatibility document actually states that the History Server is [a stable REST API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], which effectively means that ATSv1 has already been declared as a stable API. Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[
https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated YARN-3539:
-
Attachment: YARN-3539-006.patch
Patch -006 also uprates all of the data structures and {{TimelineClient}} from
{{@Unstable}} to {{@Evolving}}
Compatibility doc to state that ATS v1 is a stable REST API
---
Key: YARN-3539
URL: https://issues.apache.org/jira/browse/YARN-3539
Project: Hadoop YARN
Issue Type: Improvement
Components: documentation
Affects Versions: 2.7.0
Reporter: Steve Loughran
Assignee: Steve Loughran
Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch,
YARN-3539-003.patch, YARN-3539-004.patch, YARN-3539-005.patch,
YARN-3539-006.patch
The ATS v2 discussion and YARN-2423 have raised the question: how stable are
the ATSv1 APIs?
The existing compatibility document actually states that the History Server
is [a stable REST
API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs],
which effectively means that ATSv1 has already been declared as a stable API.
Clarify this by patching the compatibility document appropriately
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[ https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3539: -- Attachment: timeline_get_api_examples.txt I attached a document of the timeline APIs' output examples. We can make use of it for the documentation. Compatibility doc to state that ATS v1 is a stable REST API --- Key: YARN-3539 URL: https://issues.apache.org/jira/browse/YARN-3539 Project: Hadoop YARN Issue Type: Improvement Components: documentation Affects Versions: 2.7.0 Reporter: Steve Loughran Assignee: Steve Loughran Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch, YARN-3539-003.patch, YARN-3539-004.patch, YARN-3539-005.patch, YARN-3539-006.patch, timeline_get_api_examples.txt The ATS v2 discussion and YARN-2423 have raised the question: how stable are the ATSv1 APIs? The existing compatibility document actually states that the History Server is [a stable REST API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs], which effectively means that ATSv1 has already been declared as a stable API. Clarify this by patching the compatibility document appropriately -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[
https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated YARN-3539:
-
Attachment: YARN-3539-004.patch
Patch -004
this merges in YARN-1876, converting that to Markdown and replacing anything in
patch -003 which is covered in YARN-1876.
The only content retained from the 003 patch is
# error codes
# domains
# a reference (but no detail) on the response to a timeline entity POST.
Because this patch now contains YARN-1876, with its examples, it is a lot more
thorough.
One area that is not covered in the ATSv1 API is *what constitutes a valid
entity type or domain?*. This is important, as the API documentation implies
that a/b is a valid ID...it may be accepted as such, but to do so would be an
error.
There is also the fact that the {{/domain}} path was added under
{{/ws/v1/timeline/}}, so matches the path of entity types. Can you have an
entity type called domain? Was it previously possible?
I would propose
# strictly defining what constitutes a valid entity type via a regular
expression, and declaring whether the types are case sensitive.
# declaring whether {{domain}} is a valid entity type
Apart from {{/domain}}, there doesn't appear to be any restriction on entity
type names. Any new path added under the {{/ws/v1/timeline/}} is therefore at
risk of clashing with an applications use of ATS. Even a simple {{about/}} path
is potentially backwards-incompatible, so cannot be added if the API is to be
declared stable.
Either a new path is used for extensions, or a once-and-for-all subpath is
defined for extra attributes, e.g. {{/ws/v1/timeline/state}}, and that subpath
(here {{state}}) declared to be an invalid entity type.
Compatibility doc to state that ATS v1 is a stable REST API
---
Key: YARN-3539
URL: https://issues.apache.org/jira/browse/YARN-3539
Project: Hadoop YARN
Issue Type: Improvement
Components: documentation
Affects Versions: 2.7.0
Reporter: Steve Loughran
Assignee: Steve Loughran
Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch,
YARN-3539-003.patch, YARN-3539-004.patch
The ATS v2 discussion and YARN-2423 have raised the question: how stable are
the ATSv1 APIs?
The existing compatibility document actually states that the History Server
is [a stable REST
API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs],
which effectively means that ATSv1 has already been declared as a stable API.
Clarify this by patching the compatibility document appropriately
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3539) Compatibility doc to state that ATS v1 is a stable REST API
[
https://issues.apache.org/jira/browse/YARN-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated YARN-3539:
-
Attachment: YARN-3539-003.patch
Patch -003 updates {{timelineserver.md}}
# specify the REST API (non-normative)
# add some more on futures
# review configuration options
# fix up broken internal links by adding the anchors
# yarn/index.html includes link to ATS Rest API as one of the listed REST APIs
Compatibility doc to state that ATS v1 is a stable REST API
---
Key: YARN-3539
URL: https://issues.apache.org/jira/browse/YARN-3539
Project: Hadoop YARN
Issue Type: Improvement
Components: documentation
Affects Versions: 2.7.0
Reporter: Steve Loughran
Assignee: Steve Loughran
Attachments: HADOOP-11826-001.patch, HADOOP-11826-002.patch,
YARN-3539-003.patch
The ATS v2 discussion and YARN-2423 have raised the question: how stable are
the ATSv1 APIs?
The existing compatibility document actually states that the History Server
is [a stable REST
API|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#REST_APIs],
which effectively means that ATSv1 has already been declared as a stable API.
Clarify this by patching the compatibility document appropriately
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
