[jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers
[ https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345321#comment-14345321 ] Vrushali C commented on YARN-3031: -- Yes, we've decided to have only a write and aggregate api in the writer interface. The addEvent and updateMetrics is not needed, we can use the write api to do this. Also, I have the distributed shell test case working end to end for timeline v2. I have some feedback from a chat with Zhijie on that. I am updating the code and I should be posting a patch for YARN-3167, YARN-3031 and YARN-3264 today. [Storage abstraction] Create backing storage write interface for ATS writers Key: YARN-3031 URL: https://issues.apache.org/jira/browse/YARN-3031 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Sangjin Lee Assignee: Vrushali C Attachments: Sequence_diagram_write_interaction.2.png, Sequence_diagram_write_interaction.png, YARN-3031.01.patch, YARN-3031.02.patch, YARN-3031.03.patch Per design in YARN-2928, come up with the interface for the ATS writer to write to various backing storages. The interface should be created to capture the right level of abstractions so that it will enable all backing storage implementations to implement it efficiently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers
[ https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345345#comment-14345345 ] Junping Du commented on YARN-3031: -- Hi [~vrushalic], Awesome! I will help to review this patch here when patch is ready. Thanks for updating. [Storage abstraction] Create backing storage write interface for ATS writers Key: YARN-3031 URL: https://issues.apache.org/jira/browse/YARN-3031 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Sangjin Lee Assignee: Vrushali C Attachments: Sequence_diagram_write_interaction.2.png, Sequence_diagram_write_interaction.png, YARN-3031.01.patch, YARN-3031.02.patch, YARN-3031.03.patch Per design in YARN-2928, come up with the interface for the ATS writer to write to various backing storages. The interface should be created to capture the right level of abstractions so that it will enable all backing storage implementations to implement it efficiently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers
[ https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346202#comment-14346202 ] Vrushali C commented on YARN-3031: -- The writer interface is being added in the patch in https://issues.apache.org/jira/browse/YARN-3264. [Storage abstraction] Create backing storage write interface for ATS writers Key: YARN-3031 URL: https://issues.apache.org/jira/browse/YARN-3031 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Sangjin Lee Assignee: Vrushali C Attachments: Sequence_diagram_write_interaction.2.png, Sequence_diagram_write_interaction.png, YARN-3031.01.patch, YARN-3031.02.patch, YARN-3031.03.patch Per design in YARN-2928, come up with the interface for the ATS writer to write to various backing storages. The interface should be created to capture the right level of abstractions so that it will enable all backing storage implementations to implement it efficiently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers
[ https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345160#comment-14345160 ] Junping Du commented on YARN-3031: -- Discussed offline with [~vrushalic] and [~zjshen] last week and we agree to consolidate APIs here. [~vrushalic], mind giving a quick update? Thx! [Storage abstraction] Create backing storage write interface for ATS writers Key: YARN-3031 URL: https://issues.apache.org/jira/browse/YARN-3031 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Sangjin Lee Assignee: Vrushali C Attachments: Sequence_diagram_write_interaction.2.png, Sequence_diagram_write_interaction.png, YARN-3031.01.patch, YARN-3031.02.patch, YARN-3031.03.patch Per design in YARN-2928, come up with the interface for the ATS writer to write to various backing storages. The interface should be created to capture the right level of abstractions so that it will enable all backing storage implementations to implement it efficiently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers
[ https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14338654#comment-14338654 ] Junping Du commented on YARN-3031: -- bq. The AggregateUpTo enum has the tracks to aggregate along, the TimelineEntityType enum has the types of entities that can exist. There may not be aggregations along all entity types. I see. Thanks [~vrushalic] for explanation here. However, CLUSTER seems to be missing here as it is an aggregation of FLOWs. Isn't it? bq. The reasoning behind having two more apis for writing metrics and events in addition to the entity write is that, it would be good (efficient) to have the option to write a single metric or a single event. For example, say a job has many custom metrics and one particular metric is updated extremely frequently but not the others. We may want to write out only that particular metric without having to look through/write all other metrics and other information in that entity. Similarly for events. Perhaps we could do it differently that what is proposed in the patch, but the functionality of writing them individually would help in performance I believe. Agree that we should have separated interfaces to write single data entry quickly and aggregate data entries. Also some aggregator (like RM) won't even call aggregation interface here (according to YARN-3167). IMO, it sounds like two interfaces are good enough so we can merge addEvent() and updateMetrics() into a single data entry writer which can accept more generic type? That will make interface more concisely and hiding more details that could be changed in future. Thoughts? [Storage abstraction] Create backing storage write interface for ATS writers Key: YARN-3031 URL: https://issues.apache.org/jira/browse/YARN-3031 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Sangjin Lee Assignee: Vrushali C Attachments: Sequence_diagram_write_interaction.2.png, Sequence_diagram_write_interaction.png, YARN-3031.01.patch, YARN-3031.02.patch, YARN-3031.03.patch Per design in YARN-2928, come up with the interface for the ATS writer to write to various backing storages. The interface should be created to capture the right level of abstractions so that it will enable all backing storage implementations to implement it efficiently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers
[
https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14337008#comment-14337008
]
Hadoop QA commented on YARN-3031:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12700828/YARN-3031.03.patch
against trunk revision 5731c0e.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6738//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6738//console
This message is automatically generated.
[Storage abstraction] Create backing storage write interface for ATS writers
Key: YARN-3031
URL: https://issues.apache.org/jira/browse/YARN-3031
Project: Hadoop YARN
Issue Type: Sub-task
Components: timelineserver
Reporter: Sangjin Lee
Assignee: Vrushali C
Attachments: Sequence_diagram_write_interaction.2.png,
Sequence_diagram_write_interaction.png, YARN-3031.01.patch,
YARN-3031.02.patch, YARN-3031.03.patch
Per design in YARN-2928, come up with the interface for the ATS writer to
write to various backing storages. The interface should be created to capture
the right level of abstractions so that it will enable all backing storage
implementations to implement it efficiently.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers
[
https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14336994#comment-14336994
]
Junping Du commented on YARN-3031:
--
Hi [~vrushalic], thanks for updating the patch. A quick question here:
Looks like we only specify three tracks for entity: FLOW, USER and QUEUE
{code}
+public enum AggregateUpTo {
+ FLOW,
+ USER,
+ QUEUE
+}
{code}
But we have more tracks (type) in YARN-3041.
{code}
+public enum TimelineEntityType {
+ YARN_CLUSTER,
+ YARN_FLOW,
+ YARN_APPLICATION,
+ YARN_APPLICATION_ATTEMPT,
+ YARN_CONTAINER,
+ YARN_USER,
+ YARN_QUEUE;
{code}
May be we should add more here?
[Storage abstraction] Create backing storage write interface for ATS writers
Key: YARN-3031
URL: https://issues.apache.org/jira/browse/YARN-3031
Project: Hadoop YARN
Issue Type: Sub-task
Components: timelineserver
Reporter: Sangjin Lee
Assignee: Vrushali C
Attachments: Sequence_diagram_write_interaction.2.png,
Sequence_diagram_write_interaction.png, YARN-3031.01.patch,
YARN-3031.02.patch, YARN-3031.03.patch
Per design in YARN-2928, come up with the interface for the ATS writer to
write to various backing storages. The interface should be created to capture
the right level of abstractions so that it will enable all backing storage
implementations to implement it efficiently.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers
[ https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14337148#comment-14337148 ] Vinod Kumar Vavilapalli commented on YARN-3031: --- Quick comment: Following the proposal at YARN-3166, let's put the storage APIs in hadoop-yarn-server/hadoop-yarn-server-timelineservice under package o.a.h.yarn.server.timelineservice.storage? We shouldn't be exposing these to application writers as part of the hadoop-yarn-api records. [Storage abstraction] Create backing storage write interface for ATS writers Key: YARN-3031 URL: https://issues.apache.org/jira/browse/YARN-3031 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Sangjin Lee Assignee: Vrushali C Attachments: Sequence_diagram_write_interaction.2.png, Sequence_diagram_write_interaction.png, YARN-3031.01.patch, YARN-3031.02.patch, YARN-3031.03.patch Per design in YARN-2928, come up with the interface for the ATS writer to write to various backing storages. The interface should be created to capture the right level of abstractions so that it will enable all backing storage implementations to implement it efficiently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers
[ https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14337401#comment-14337401 ] Vrushali C commented on YARN-3031: -- Hi [~djp] The AggregateUpTo enum has the tracks to aggregate along, the TimelineEntityType enum has the types of entities that can exist. There may not be aggregations along all entity types. For example: a query can be, give all the apps run by this user in the last week. This will read the data that is aggregated along the USER track. I think I can rename those with a YARN prefix though. [Storage abstraction] Create backing storage write interface for ATS writers Key: YARN-3031 URL: https://issues.apache.org/jira/browse/YARN-3031 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Sangjin Lee Assignee: Vrushali C Attachments: Sequence_diagram_write_interaction.2.png, Sequence_diagram_write_interaction.png, YARN-3031.01.patch, YARN-3031.02.patch, YARN-3031.03.patch Per design in YARN-2928, come up with the interface for the ATS writer to write to various backing storages. The interface should be created to capture the right level of abstractions so that it will enable all backing storage implementations to implement it efficiently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers
[ https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14337393#comment-14337393 ] Vrushali C commented on YARN-3031: -- Yes, will update it in the next patch. [Storage abstraction] Create backing storage write interface for ATS writers Key: YARN-3031 URL: https://issues.apache.org/jira/browse/YARN-3031 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Sangjin Lee Assignee: Vrushali C Attachments: Sequence_diagram_write_interaction.2.png, Sequence_diagram_write_interaction.png, YARN-3031.01.patch, YARN-3031.02.patch, YARN-3031.03.patch Per design in YARN-2928, come up with the interface for the ATS writer to write to various backing storages. The interface should be created to capture the right level of abstractions so that it will enable all backing storage implementations to implement it efficiently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers
[ https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14336054#comment-14336054 ] Vrushali C commented on YARN-3031: -- Hi [~zjshen], Thanks for the prompt review, appreciate it! These are very good points you mention, let me add some more context around why these are coded this way right now. 1. The reasoning behind having two more apis for writing metrics and events in addition to the entity write is that, it would be good (efficient) to have the option to write a single metric or a single event. For example, say a job has many custom metrics and one particular metric is updated extremely frequently but not the others. We may want to write out only that particular metric without having to look through/write all other metrics and other information in that entity. Similarly for events. Perhaps we could do it differently that what is proposed in the patch, but the functionality of writing them individually would help in performance I believe. 2. Having a separate write and aggregator API makes them independent of the order in which the entity details and aggregation are invoked/stored and makes them independent of each other. For instance, we may choose to invoke the aggregation at a different frequency (more slower) than the regular entity writes. Hence two apis. 3. The TimelineServiceWriteResponse has two error codes presently: NO_START_TIME and IO_EXCEPTION. We can of course add in more error codes as we proceed. The reason I chose these two for now is that each flow is inherently associated with a submit timestamp (run id of that flow). In case, we don’t find that timestamp, it would be difficult to write the flow information for that run to the store - I think an error should be thrown with an error code. The other one, IO_EXCEPTION is what I thought would help accounting for write/put errors to the store- we should be able to indicate that the write did not go through. We can rename these if these names don’t sound meaningful. thanks Vrushali [Storage abstraction] Create backing storage write interface for ATS writers Key: YARN-3031 URL: https://issues.apache.org/jira/browse/YARN-3031 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Sangjin Lee Assignee: Vrushali C Attachments: Sequence_diagram_write_interaction.2.png, Sequence_diagram_write_interaction.png, YARN-3031.01.patch, YARN-3031.02.patch Per design in YARN-2928, come up with the interface for the ATS writer to write to various backing storages. The interface should be created to capture the right level of abstractions so that it will enable all backing storage implementations to implement it efficiently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers
[
https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14335756#comment-14335756
]
Hadoop QA commented on YARN-3031:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12700617/YARN-3031.02.patch
against trunk revision ac3468a.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6717//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6717//console
This message is automatically generated.
[Storage abstraction] Create backing storage write interface for ATS writers
Key: YARN-3031
URL: https://issues.apache.org/jira/browse/YARN-3031
Project: Hadoop YARN
Issue Type: Sub-task
Components: timelineserver
Reporter: Sangjin Lee
Assignee: Vrushali C
Attachments: Sequence_diagram_write_interaction.2.png,
Sequence_diagram_write_interaction.png, YARN-3031.01.patch, YARN-3031.02.patch
Per design in YARN-2928, come up with the interface for the ATS writer to
write to various backing storages. The interface should be created to capture
the right level of abstractions so that it will enable all backing storage
implementations to implement it efficiently.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers
[ https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329368#comment-14329368 ] Zhijie Shen commented on YARN-3031: --- [~vrushalic], YARN-3041 is committed, would you mind updating this patch accordingly? And we'd like to review it again. Thanks! [Storage abstraction] Create backing storage write interface for ATS writers Key: YARN-3031 URL: https://issues.apache.org/jira/browse/YARN-3031 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Sangjin Lee Assignee: Vrushali C Attachments: Sequence_diagram_write_interaction.2.png, Sequence_diagram_write_interaction.png, YARN-3031.01.patch Per design in YARN-2928, come up with the interface for the ATS writer to write to various backing storages. The interface should be created to capture the right level of abstractions so that it will enable all backing storage implementations to implement it efficiently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers
[ https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329375#comment-14329375 ] Vrushali C commented on YARN-3031: -- Yes, I will.. [Storage abstraction] Create backing storage write interface for ATS writers Key: YARN-3031 URL: https://issues.apache.org/jira/browse/YARN-3031 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Sangjin Lee Assignee: Vrushali C Attachments: Sequence_diagram_write_interaction.2.png, Sequence_diagram_write_interaction.png, YARN-3031.01.patch Per design in YARN-2928, come up with the interface for the ATS writer to write to various backing storages. The interface should be created to capture the right level of abstractions so that it will enable all backing storage implementations to implement it efficiently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers
[
https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318758#comment-14318758
]
Hadoop QA commented on YARN-3031:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12698488/Sequence_diagram_write_interaction.2.png
against trunk revision 9e33c99.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6618//console
This message is automatically generated.
[Storage abstraction] Create backing storage write interface for ATS writers
Key: YARN-3031
URL: https://issues.apache.org/jira/browse/YARN-3031
Project: Hadoop YARN
Issue Type: Sub-task
Components: timelineserver
Reporter: Sangjin Lee
Assignee: Vrushali C
Attachments: Sequence_diagram_write_interaction.2.png,
Sequence_diagram_write_interaction.png, YARN-3031.01.patch
Per design in YARN-2928, come up with the interface for the ATS writer to
write to various backing storages. The interface should be created to capture
the right level of abstractions so that it will enable all backing storage
implementations to implement it efficiently.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers
[ https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318680#comment-14318680 ] Vrushali C commented on YARN-3031: -- hi [~djp], Thanks for looking at this! Yes, the calls from client (like AM) to write an entity info could be sync ( blocking) or async (non-blocking), but the storage writer api (TimelineServiceWriter) calls are always synchronous (blocking). The Base Aggregator Service would provide both options of writes (sync and async) in it's API. Would be part of YARN-3167 I think. I have modified the sequence diagram to say that calls from AM to Base Aggregator Service can be sync or async. thanks Vrushali [Storage abstraction] Create backing storage write interface for ATS writers Key: YARN-3031 URL: https://issues.apache.org/jira/browse/YARN-3031 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Sangjin Lee Assignee: Vrushali C Attachments: Sequence_diagram_write_interaction.png, YARN-3031.01.patch Per design in YARN-2928, come up with the interface for the ATS writer to write to various backing storages. The interface should be created to capture the right level of abstractions so that it will enable all backing storage implementations to implement it efficiently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers
[ https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318576#comment-14318576 ] Junping Du commented on YARN-3031: -- Thanks [~vrushalic] for updating the patch and sequence diagram here! One quick question about sequence diagram: looks like we are doing async call (non-blocking call) between AM and BaseAggregatorService here. Do have some scenarios that AM write some entities/event and check them immediately? If so, we may need to find some way to notify AM about the result of writing (it may get failed, so AM can redo it later) or we also provide a blocking call option? [Storage abstraction] Create backing storage write interface for ATS writers Key: YARN-3031 URL: https://issues.apache.org/jira/browse/YARN-3031 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Sangjin Lee Assignee: Vrushali C Attachments: Sequence_diagram_write_interaction.png, YARN-3031.01.patch Per design in YARN-2928, come up with the interface for the ATS writer to write to various backing storages. The interface should be created to capture the right level of abstractions so that it will enable all backing storage implementations to implement it efficiently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers
[
https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14317442#comment-14317442
]
Hadoop QA commented on YARN-3031:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12698250/Sequence_diagram_write_interaction.png
against trunk revision 67efab9.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6603//console
This message is automatically generated.
[Storage abstraction] Create backing storage write interface for ATS writers
Key: YARN-3031
URL: https://issues.apache.org/jira/browse/YARN-3031
Project: Hadoop YARN
Issue Type: Sub-task
Components: timelineserver
Reporter: Sangjin Lee
Assignee: Vrushali C
Attachments: Sequence_diagram_write_interaction.png,
YARN-3031.01.patch
Per design in YARN-2928, come up with the interface for the ATS writer to
write to various backing storages. The interface should be created to capture
the right level of abstractions so that it will enable all backing storage
implementations to implement it efficiently.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
