[jira] [Updated] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-5094: - Component/s: (was: timelineserver) > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.9.0, 3.0.0 >Reporter: Sangjin Lee >Assignee: Haibo Chen >Priority: Critical > Labels: YARN-5355 > Fix For: 3.1.0, 2.9.1, 3.0.1 > > Attachments: YARN-5094-YARN-2928.001.patch, YARN-5094.00.patch, > YARN-5094.02.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-5094: -- Affects Version/s: (was: YARN-2928) 2.9.0 3.0.0 > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: 2.9.0, 3.0.0 >Reporter: Sangjin Lee >Assignee: Haibo Chen >Priority: Critical > Labels: YARN-5355 > Attachments: YARN-5094-YARN-2928.001.patch, YARN-5094.00.patch, > YARN-5094.02.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-5094: -- Priority: Critical (was: Major) > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: 2.9.0, 3.0.0 >Reporter: Sangjin Lee >Assignee: Haibo Chen >Priority: Critical > Labels: YARN-5355 > Attachments: YARN-5094-YARN-2928.001.patch, YARN-5094.00.patch, > YARN-5094.02.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vrushali C updated YARN-5094: - Parent Issue: YARN-7055 (was: YARN-5355) > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Haibo Chen > Labels: YARN-5355 > Attachments: YARN-5094.00.patch, YARN-5094.02.patch, > YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-5094: - Attachment: YARN-5094.02.patch Patch updated to ignore ApplicationLocalizationEvent timestamp when matching arguments > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Haibo Chen > Labels: YARN-5355 > Attachments: YARN-5094.00.patch, YARN-5094.02.patch, > YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-5094: - Attachment: YARN-5094.00.patch > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Haibo Chen > Labels: YARN-5355 > Attachments: YARN-5094.00.patch, YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5094) some YARN container events have timestamp of -1
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-5094: - Summary: some YARN container events have timestamp of -1 (was: some YARN container events have timestamp of -1 in REST output) > some YARN container events have timestamp of -1 > --- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Haibo Chen > Labels: YARN-5355 > Attachments: YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5094) some YARN container events have timestamp of -1 in REST output
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joep Rottinghuis updated YARN-5094: --- Parent Issue: YARN-5355 (was: YARN-2928) > some YARN container events have timestamp of -1 in REST output > -- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Li Lu > Labels: YARN-5355 > Attachments: YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5094) some YARN container events have timestamp of -1 in REST output
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joep Rottinghuis updated YARN-5094: --- Labels: YARN-5355 (was: ) > some YARN container events have timestamp of -1 in REST output > -- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Li Lu > Labels: YARN-5355 > Attachments: YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5094) some YARN container events have timestamp of -1 in REST output
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sangjin Lee updated YARN-5094: -- Labels: (was: yarn-2928-1st-milestone) > some YARN container events have timestamp of -1 in REST output > -- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Li Lu > Attachments: YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5094) some YARN container events have timestamp of -1 in REST output
[ https://issues.apache.org/jira/browse/YARN-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Lu updated YARN-5094: Attachment: YARN-5094-YARN-2928.001.patch First draft to resolve the problem. Note that I still decide to put timestamp data in events. I think this is fine since creating the timestamp itself will not introduce much overhead to node managers. This is very different to the RM's use cases (where we may create a lot of events within one second). I have not yet added any unit tests for this specific fix. In long term, we may want to use DistributedShell as a tool to test other part of ATS. > some YARN container events have timestamp of -1 in REST output > -- > > Key: YARN-5094 > URL: https://issues.apache.org/jira/browse/YARN-5094 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Li Lu > Labels: yarn-2928-1st-milestone > Attachments: YARN-5094-YARN-2928.001.patch > > > Some events in the YARN container entities have timestamp of -1. The > RM-generated container events have proper timestamps. It appears that it's > the NM-generated events that have -1: YARN_CONTAINER_CREATED, > YARN_CONTAINER_FINISHED, YARN_NM_CONTAINER_LOCALIZATION_FINISHED, > YARN_NM_CONTAINER_LOCALIZATION_STARTED. > In the YARN container page, > {noformat} > { > id: "YARN_CONTAINER_CREATED", > timestamp: -1, > info: { } > }, > { > id: "YARN_CONTAINER_FINISHED", > timestamp: -1, > info: { > YARN_CONTAINER_EXIT_STATUS: 0, > YARN_CONTAINER_STATE: "RUNNING", > YARN_CONTAINER_DIAGNOSTICS_INFO: "" > } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_FINISHED", > timestamp: -1, > info: { } > }, > { > id: "YARN_NM_CONTAINER_LOCALIZATION_STARTED", > timestamp: -1, > info: { } > } > {noformat} > I think the data itself is OK, but the values are not being populated in the > REST output? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org