[jira] [Commented] (YARN-3264) [Storage implementation] Create backing storage write interface and a POC only file based storage implementation
[ https://issues.apache.org/jira/browse/YARN-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15369719#comment-15369719 ] Hudson commented on YARN-3264: -- SUCCESS: Integrated in Hadoop-trunk-Commit #10074 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/10074/]) YARN-3264. Created backing storage write interface and a POC only FS (sjlee: rev f0e752c14b0b441955cdfc94f95937c90f1f055d) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/TimelineWriteResponse.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/TimelineWriter.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/aggregator/TestTimelineAggregator.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/TestFileSystemTimelineWriterImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/TimelineAggregationTrack.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/FileSystemTimelineWriterImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/aggregator/TimelineAggregator.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java > [Storage implementation] Create backing storage write interface and a POC > only file based storage implementation > - > > Key: YARN-3264 > URL: https://issues.apache.org/jira/browse/YARN-3264 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Vrushali C >Assignee: Vrushali C > Fix For: YARN-2928 > > Attachments: YARN-3264.001.patch, YARN-3264.002.patch, > YARN-3264.003.patch, YARN-3264.004.patch, YARN-3264.005.patch, > YARN-3264.006.patch, YARN-3264.007.patch, YARN-3264.008.patch > > > For the PoC, need to create a backend impl for file based storage of entities -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3264) [Storage implementation] Create backing storage write interface and a POC only file based storage implementation
[ https://issues.apache.org/jira/browse/YARN-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350695#comment-14350695 ] Vinod Kumar Vavilapalli commented on YARN-3264: --- Congrats on your first patch, Vrushali! Great progress everyone! [Storage implementation] Create backing storage write interface and a POC only file based storage implementation - Key: YARN-3264 URL: https://issues.apache.org/jira/browse/YARN-3264 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Vrushali C Assignee: Vrushali C Fix For: YARN-2928 Attachments: YARN-3264.001.patch, YARN-3264.002.patch, YARN-3264.003.patch, YARN-3264.004.patch, YARN-3264.005.patch, YARN-3264.006.patch, YARN-3264.007.patch, YARN-3264.008.patch For the PoC, need to create a backend impl for file based storage of entities -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3264) [Storage implementation] Create backing storage write interface and a POC only file based storage implementation
[ https://issues.apache.org/jira/browse/YARN-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349352#comment-14349352 ] Li Lu commented on YARN-3264: - Thanks [~vrushalic]! Applied the v6 patch and tested locally, diffed with the v7 patch and the latest one (v7) LGTM. [Storage implementation] Create backing storage write interface and a POC only file based storage implementation - Key: YARN-3264 URL: https://issues.apache.org/jira/browse/YARN-3264 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Vrushali C Assignee: Vrushali C Attachments: YARN-3264.001.patch, YARN-3264.002.patch, YARN-3264.003.patch, YARN-3264.004.patch, YARN-3264.005.patch, YARN-3264.006.patch, YARN-3264.007.patch For the PoC, need to create a backend impl for file based storage of entities -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3264) [Storage implementation] Create backing storage write interface and a POC only file based storage implementation
[ https://issues.apache.org/jira/browse/YARN-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349356#comment-14349356 ] Hadoop QA commented on YARN-3264: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12702885/YARN-3264.007.patch against trunk revision 952640f. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6868//console This message is automatically generated. [Storage implementation] Create backing storage write interface and a POC only file based storage implementation - Key: YARN-3264 URL: https://issues.apache.org/jira/browse/YARN-3264 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Vrushali C Assignee: Vrushali C Attachments: YARN-3264.001.patch, YARN-3264.002.patch, YARN-3264.003.patch, YARN-3264.004.patch, YARN-3264.005.patch, YARN-3264.006.patch, YARN-3264.007.patch For the PoC, need to create a backend impl for file based storage of entities -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3264) [Storage implementation] Create backing storage write interface and a POC only file based storage implementation
[ https://issues.apache.org/jira/browse/YARN-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349504#comment-14349504 ] Robert Kanter commented on YARN-3264: - Looks good. Two minor thing: - {{TIMELINE_SERVICE_STORAGAGE_EXTENSION}} in {{FileSystemTimelineWriterImpl}} has a typo. - Would it make sense to rename {{TimelineAggregationGranularity}} to {{TimelineAggregationTrack}}? That seems more clear about what it's for, and more in line with what all the comments say about it. [Storage implementation] Create backing storage write interface and a POC only file based storage implementation - Key: YARN-3264 URL: https://issues.apache.org/jira/browse/YARN-3264 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Vrushali C Assignee: Vrushali C Attachments: YARN-3264.001.patch, YARN-3264.002.patch, YARN-3264.003.patch, YARN-3264.004.patch, YARN-3264.005.patch, YARN-3264.006.patch, YARN-3264.007.patch For the PoC, need to create a backend impl for file based storage of entities -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3264) [Storage implementation] Create backing storage write interface and a POC only file based storage implementation
[ https://issues.apache.org/jira/browse/YARN-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349578#comment-14349578 ] Zhijie Shen commented on YARN-3264: --- +1 LGTM, will commit the patch [Storage implementation] Create backing storage write interface and a POC only file based storage implementation - Key: YARN-3264 URL: https://issues.apache.org/jira/browse/YARN-3264 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Vrushali C Assignee: Vrushali C Attachments: YARN-3264.001.patch, YARN-3264.002.patch, YARN-3264.003.patch, YARN-3264.004.patch, YARN-3264.005.patch, YARN-3264.006.patch, YARN-3264.007.patch, YARN-3264.008.patch For the PoC, need to create a backend impl for file based storage of entities -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3264) [Storage implementation] Create backing storage write interface and a POC only file based storage implementation
[ https://issues.apache.org/jira/browse/YARN-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14347460#comment-14347460 ] Varun Saxena commented on YARN-3264: We can probably use try with resources construct as well. [Storage implementation] Create backing storage write interface and a POC only file based storage implementation - Key: YARN-3264 URL: https://issues.apache.org/jira/browse/YARN-3264 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Vrushali C Assignee: Vrushali C Attachments: YARN-3264.001.patch, YARN-3264.002.patch, YARN-3264.003.patch For the PoC, need to create a backend impl for file based storage of entities -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3264) [Storage implementation] Create backing storage write interface and a POC only file based storage implementation
[ https://issues.apache.org/jira/browse/YARN-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14347387#comment-14347387 ] Li Lu commented on YARN-3264: - Hi [~vrushalic], thanks for the patch! In general it looks good to me. I have a few quick questions about it: # In the following lines: {code} +String tmpRoot = FileSystemTimelineServiceWriterImpl.TIMELINE_SERVICE_STORAGE_DIR_ROOT; +if (tmpRoot == null || tmpRoot.isEmpty()) { + tmpRoot = /tmp/; +} {code} TIMELINE_SERVICE_STORAGE_DIR_ROOT is defined as final in FileSystemTimelineServiceWriterImpl (with a not-null initial value), why are we still checking if it's null here? (Am I missing anything? ) # Why we're removing the abstract keyword for the TimelineAggregator class? I thought this class was supposed to be abstract? {code} -public abstract class TimelineAggregator extends CompositeService { +public class TimelineAggregator extends CompositeService { {code} [Storage implementation] Create backing storage write interface and a POC only file based storage implementation - Key: YARN-3264 URL: https://issues.apache.org/jira/browse/YARN-3264 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Vrushali C Assignee: Vrushali C Attachments: YARN-3264.001.patch, YARN-3264.002.patch, YARN-3264.003.patch For the PoC, need to create a backend impl for file based storage of entities -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3264) [Storage implementation] Create backing storage write interface and a POC only file based storage implementation
[ https://issues.apache.org/jira/browse/YARN-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14347490#comment-14347490 ] Hadoop QA commented on YARN-3264: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12702599/YARN-3264.004.patch against trunk revision ed70fa1. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6846//console This message is automatically generated. [Storage implementation] Create backing storage write interface and a POC only file based storage implementation - Key: YARN-3264 URL: https://issues.apache.org/jira/browse/YARN-3264 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Vrushali C Assignee: Vrushali C Attachments: YARN-3264.001.patch, YARN-3264.002.patch, YARN-3264.003.patch, YARN-3264.004.patch For the PoC, need to create a backend impl for file based storage of entities -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3264) [Storage implementation] Create backing storage write interface and a POC only file based storage implementation
[ https://issues.apache.org/jira/browse/YARN-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14347626#comment-14347626 ] Li Lu commented on YARN-3264: - Hi [~vrushalic], thanks for the update. Unfortunately the TestTimelineAggregator part failed to compile on my machine, due to the abstract TimelineAggregator class. Here, to test the basic features of TimelineAggregators, maybe we'd like to set up a SimpleTimelineAggregator class that only inherits TimelineAggregator but performs nothing else, and use it in TestTimelineAggregator? Also, I briefly skimmed through the patch and there are some unused imports. Maybe we would like to do a final cleanup before it's committed? (It's quite simple with an IDE, so let's put that to the final round. ) Thanks! [Storage implementation] Create backing storage write interface and a POC only file based storage implementation - Key: YARN-3264 URL: https://issues.apache.org/jira/browse/YARN-3264 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Vrushali C Assignee: Vrushali C Attachments: YARN-3264.001.patch, YARN-3264.002.patch, YARN-3264.003.patch, YARN-3264.004.patch For the PoC, need to create a backend impl for file based storage of entities -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3264) [Storage implementation] Create backing storage write interface and a POC only file based storage implementation
[ https://issues.apache.org/jira/browse/YARN-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14347641#comment-14347641 ] Vrushali C commented on YARN-3264: -- [~gtCarrera] thanks! Will fix that test and remove the unused imports in the next update. [Storage implementation] Create backing storage write interface and a POC only file based storage implementation - Key: YARN-3264 URL: https://issues.apache.org/jira/browse/YARN-3264 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Vrushali C Assignee: Vrushali C Attachments: YARN-3264.001.patch, YARN-3264.002.patch, YARN-3264.003.patch, YARN-3264.004.patch For the PoC, need to create a backend impl for file based storage of entities -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3264) [Storage implementation] Create backing storage write interface and a POC only file based storage implementation
[ https://issues.apache.org/jira/browse/YARN-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348052#comment-14348052 ] Zhijie Shen commented on YARN-3264: --- The patch looks almost good. Two minor issue: 1. Should we still keep TimelineAggregator abstract? 2. To be consistent, how about renaming: TimelineServiceWriteResponse - TimelineWriteResponse, TimelineServiceWriteError -TimelineWriteResponse, and TimelineServiceWriter(Impl) - TimelineWriter(Impl) and so on [Storage implementation] Create backing storage write interface and a POC only file based storage implementation - Key: YARN-3264 URL: https://issues.apache.org/jira/browse/YARN-3264 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Vrushali C Assignee: Vrushali C Attachments: YARN-3264.001.patch, YARN-3264.002.patch, YARN-3264.003.patch, YARN-3264.004.patch, YARN-3264.005.patch For the PoC, need to create a backend impl for file based storage of entities -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3264) [Storage implementation] Create backing storage write interface and a POC only file based storage implementation
[ https://issues.apache.org/jira/browse/YARN-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14347893#comment-14347893 ] Hadoop QA commented on YARN-3264: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12702658/YARN-3264.005.patch against trunk revision 722b479. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6855//console This message is automatically generated. [Storage implementation] Create backing storage write interface and a POC only file based storage implementation - Key: YARN-3264 URL: https://issues.apache.org/jira/browse/YARN-3264 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Vrushali C Assignee: Vrushali C Attachments: YARN-3264.001.patch, YARN-3264.002.patch, YARN-3264.003.patch, YARN-3264.004.patch, YARN-3264.005.patch For the PoC, need to create a backend impl for file based storage of entities -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3264) [Storage implementation] Create backing storage write interface and a POC only file based storage implementation
[ https://issues.apache.org/jira/browse/YARN-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348228#comment-14348228 ] Vrushali C commented on YARN-3264: -- Hi [~gtCarrera] Huh, yes, the patch looks much smaller. I have renamed several classes in this new patch, I will look through it and regenerate it again. [Storage implementation] Create backing storage write interface and a POC only file based storage implementation - Key: YARN-3264 URL: https://issues.apache.org/jira/browse/YARN-3264 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Vrushali C Assignee: Vrushali C Attachments: YARN-3264.001.patch, YARN-3264.002.patch, YARN-3264.003.patch, YARN-3264.004.patch, YARN-3264.005.patch, YARN-3264.006.patch For the PoC, need to create a backend impl for file based storage of entities -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3264) [Storage implementation] Create backing storage write interface and a POC only file based storage implementation
[ https://issues.apache.org/jira/browse/YARN-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348137#comment-14348137 ] Li Lu commented on YARN-3264: - Hi [~vrushalic], thanks for the update! The latest patch is significantly smaller than previous ones, and there are some files missing? [Storage implementation] Create backing storage write interface and a POC only file based storage implementation - Key: YARN-3264 URL: https://issues.apache.org/jira/browse/YARN-3264 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Vrushali C Assignee: Vrushali C Attachments: YARN-3264.001.patch, YARN-3264.002.patch, YARN-3264.003.patch, YARN-3264.004.patch, YARN-3264.005.patch, YARN-3264.006.patch For the PoC, need to create a backend impl for file based storage of entities -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3264) [Storage implementation] Create backing storage write interface and a POC only file based storage implementation
[ https://issues.apache.org/jira/browse/YARN-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348143#comment-14348143 ] Hadoop QA commented on YARN-3264: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12702703/YARN-3264.006.patch against trunk revision 8d88691. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6857//console This message is automatically generated. [Storage implementation] Create backing storage write interface and a POC only file based storage implementation - Key: YARN-3264 URL: https://issues.apache.org/jira/browse/YARN-3264 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Vrushali C Assignee: Vrushali C Attachments: YARN-3264.001.patch, YARN-3264.002.patch, YARN-3264.003.patch, YARN-3264.004.patch, YARN-3264.005.patch, YARN-3264.006.patch For the PoC, need to create a backend impl for file based storage of entities -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3264) [Storage implementation] Create backing storage write interface and a POC only file based storage implementation
[ https://issues.apache.org/jira/browse/YARN-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348234#comment-14348234 ] Hadoop QA commented on YARN-3264: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12702725/YARN-3264.006.patch against trunk revision 3482080. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6862//console This message is automatically generated. [Storage implementation] Create backing storage write interface and a POC only file based storage implementation - Key: YARN-3264 URL: https://issues.apache.org/jira/browse/YARN-3264 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Vrushali C Assignee: Vrushali C Attachments: YARN-3264.001.patch, YARN-3264.002.patch, YARN-3264.003.patch, YARN-3264.004.patch, YARN-3264.005.patch, YARN-3264.006.patch For the PoC, need to create a backend impl for file based storage of entities -- This message was sent by Atlassian JIRA (v6.3.4#6332)