[jira] [Commented] (YARN-8593) Add new RM web service endpoint to get cluster user info
[ https://issues.apache.org/jira/browse/YARN-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561538#comment-16561538 ] Akhil PB commented on YARN-8593: [~sunilg] [~rohithsharma] please review the updated v2 patch, which includes test cases. > Add new RM web service endpoint to get cluster user info > > > Key: YARN-8593 > URL: https://issues.apache.org/jira/browse/YARN-8593 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8593.001.patch, YARN-8593.002.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8593) Add new RM web service endpoint to get cluster user info
[ https://issues.apache.org/jira/browse/YARN-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akhil PB updated YARN-8593: --- Attachment: YARN-8593.002.patch > Add new RM web service endpoint to get cluster user info > > > Key: YARN-8593 > URL: https://issues.apache.org/jira/browse/YARN-8593 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8593.001.patch, YARN-8593.002.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8592) [UI2] Resource Manager Web UI V2 shows a blank page in windows OS And Chrome browser
[ https://issues.apache.org/jira/browse/YARN-8592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561467#comment-16561467 ] genericqa commented on YARN-8592: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 25s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m 26s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 38m 36s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 52s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 27s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 52m 53s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 | | JIRA Issue | YARN-8592 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12933546/YARN-8592.001.patch | | Optional Tests | asflicense shadedclient | | uname | Linux 51f9fe92abae 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 0857f11 | | maven | version: Apache Maven 3.3.9 | | Max. process+thread count | 303 (vs. ulimit of 1) | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/21426/console | | Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > [UI2] Resource Manager Web UI V2 shows a blank page in windows OS And Chrome > browser > > > Key: YARN-8592 > URL: https://issues.apache.org/jira/browse/YARN-8592 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Akhil S Naik >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8592.001.patch > > > after successfully installing ambari 2.7v and hdp 3.0 in Windows Operating > Sytem . > But, When we connect to Resource Manager UI(v2), web browser shows a blank > page. > In a chrome development tool, this error is shown > Stack trace.. > {code:java} > TypeError: Cannot read property 'join's null > at new i (hostname:8088/ui2/assets/vendor.js:5598:40988) > at l (hostname:8088/ui2/assets/vendor.js:5598:41338) > at p (hostname:8088/ui2/assets/vendor.js:5598:42035) > at Function.q [as guess] (hostname:8088/ui2/assets/vendor.js:5598:42235) > at getDefaultTimezone (hostname:8088/ui2/assets/yarn-ui.js:366:445) > at convertTimestampWithT (hostname:8088/ui2/assets/yarn-ui.js:366:656) > at Object.timeStampToDate (hostname:8088/ui2/assets/yarn-ui.js:348:127) > at Class.internalNormalizeSingleResponse > (hostname:8088/ui2/assets/yarn-ui.js:287:73) > at Class. (hostname:8088/ui2/assets/yarn-ui.js:288:2024) > at Array.map (native) > {code} > The Error is hit in the underlying framework that Yarn-UI uses : > moment-timezone-with-data-2010-2020.min.js > this is the discussion happening in framework github for the same : > https://github.com/moment/moment-timezone/issues/294 > Please find the discussion on this bug here : > https://community.hortonworks.com/questions/208887/resource-manager-web-ui-v2-shows-a-blank-page.html > It works well in firefox browser in windows Operating systems. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8592) [UI2] Resource Manager Web UI V2 shows a blank page in windows OS And Chrome browser
[ https://issues.apache.org/jira/browse/YARN-8592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akhil PB updated YARN-8592: --- Attachment: (was: YARN-8592.001.patch) > [UI2] Resource Manager Web UI V2 shows a blank page in windows OS And Chrome > browser > > > Key: YARN-8592 > URL: https://issues.apache.org/jira/browse/YARN-8592 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Akhil S Naik >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8592.001.patch > > > after successfully installing ambari 2.7v and hdp 3.0 in Windows Operating > Sytem . > But, When we connect to Resource Manager UI(v2), web browser shows a blank > page. > In a chrome development tool, this error is shown > Stack trace.. > {code:java} > TypeError: Cannot read property 'join's null > at new i (hostname:8088/ui2/assets/vendor.js:5598:40988) > at l (hostname:8088/ui2/assets/vendor.js:5598:41338) > at p (hostname:8088/ui2/assets/vendor.js:5598:42035) > at Function.q [as guess] (hostname:8088/ui2/assets/vendor.js:5598:42235) > at getDefaultTimezone (hostname:8088/ui2/assets/yarn-ui.js:366:445) > at convertTimestampWithT (hostname:8088/ui2/assets/yarn-ui.js:366:656) > at Object.timeStampToDate (hostname:8088/ui2/assets/yarn-ui.js:348:127) > at Class.internalNormalizeSingleResponse > (hostname:8088/ui2/assets/yarn-ui.js:287:73) > at Class. (hostname:8088/ui2/assets/yarn-ui.js:288:2024) > at Array.map (native) > {code} > The Error is hit in the underlying framework that Yarn-UI uses : > moment-timezone-with-data-2010-2020.min.js > this is the discussion happening in framework github for the same : > https://github.com/moment/moment-timezone/issues/294 > Please find the discussion on this bug here : > https://community.hortonworks.com/questions/208887/resource-manager-web-ui-v2-shows-a-blank-page.html > It works well in firefox browser in windows Operating systems. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8592) [UI2] Resource Manager Web UI V2 shows a blank page in windows OS And Chrome browser
[ https://issues.apache.org/jira/browse/YARN-8592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akhil PB updated YARN-8592: --- Attachment: YARN-8592.001.patch > [UI2] Resource Manager Web UI V2 shows a blank page in windows OS And Chrome > browser > > > Key: YARN-8592 > URL: https://issues.apache.org/jira/browse/YARN-8592 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Akhil S Naik >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8592.001.patch > > > after successfully installing ambari 2.7v and hdp 3.0 in Windows Operating > Sytem . > But, When we connect to Resource Manager UI(v2), web browser shows a blank > page. > In a chrome development tool, this error is shown > Stack trace.. > {code:java} > TypeError: Cannot read property 'join's null > at new i (hostname:8088/ui2/assets/vendor.js:5598:40988) > at l (hostname:8088/ui2/assets/vendor.js:5598:41338) > at p (hostname:8088/ui2/assets/vendor.js:5598:42035) > at Function.q [as guess] (hostname:8088/ui2/assets/vendor.js:5598:42235) > at getDefaultTimezone (hostname:8088/ui2/assets/yarn-ui.js:366:445) > at convertTimestampWithT (hostname:8088/ui2/assets/yarn-ui.js:366:656) > at Object.timeStampToDate (hostname:8088/ui2/assets/yarn-ui.js:348:127) > at Class.internalNormalizeSingleResponse > (hostname:8088/ui2/assets/yarn-ui.js:287:73) > at Class. (hostname:8088/ui2/assets/yarn-ui.js:288:2024) > at Array.map (native) > {code} > The Error is hit in the underlying framework that Yarn-UI uses : > moment-timezone-with-data-2010-2020.min.js > this is the discussion happening in framework github for the same : > https://github.com/moment/moment-timezone/issues/294 > Please find the discussion on this bug here : > https://community.hortonworks.com/questions/208887/resource-manager-web-ui-v2-shows-a-blank-page.html > It works well in firefox browser in windows Operating systems. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8558) NM recovery level db not cleaned up properly on container finish
[ https://issues.apache.org/jira/browse/YARN-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bibin A Chundatt updated YARN-8558: --- Attachment: YARN-8558-branch-3.0.003.patch > NM recovery level db not cleaned up properly on container finish > > > Key: YARN-8558 > URL: https://issues.apache.org/jira/browse/YARN-8558 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.0.0, 3.1.0 >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > Fix For: 3.2.0, 3.1.1 > > Attachments: YARN-8558-branch-3.0.002.patch, > YARN-8558-branch-3.0.003.patch, YARN-8558.001.patch, YARN-8558.002.patch > > > {code} > 2018-07-20 16:49:23,117 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: > Application application_1531994217928_0054 transitioned from NEW to INITING > 2018-07-20 16:49:23,204 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container container_1531994217928_0001_01_18 with incomplete > records > 2018-07-20 16:49:23,204 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container container_1531994217928_0001_01_19 with incomplete > records > 2018-07-20 16:49:23,204 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container container_1531994217928_0001_01_20 with incomplete > records > 2018-07-20 16:49:23,205 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container container_1531994217928_0001_01_21 with incomplete > records > 2018-07-20 16:49:23,205 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container container_1531994217928_0001_01_22 with incomplete > records > 2018-07-20 16:49:23,205 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container container_1531994217928_0001_01_23 with incomplete > records > 2018-07-20 16:49:23,205 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container container_1531994217928_0001_01_24 with incomplete > records > 2018-07-20 16:49:23,205 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container container_1531994217928_0001_01_25 with incomplete > records > 2018-07-20 16:49:23,205 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container container_1531994217928_0001_01_38 with incomplete > records > 2018-07-20 16:49:23,205 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container container_1531994217928_0001_01_39 with incomplete > records > 2018-07-20 16:49:23,206 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container container_1531994217928_0001_01_41 with incomplete > records > 2018-07-20 16:49:23,206 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container container_1531994217928_0001_01_44 with incomplete > records > 2018-07-20 16:49:23,206 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container container_1531994217928_0001_01_46 with incomplete > records > 2018-07-20 16:49:23,206 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container container_1531994217928_0001_01_49 with incomplete > records > 2018-07-20 16:49:23,206 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container container_1531994217928_0001_01_52 with incomplete > records > 2018-07-20 16:49:23,206 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container container_1531994217928_0001_01_54 with incomplete > records > 2018-07-20 16:49:23,206 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container container_1531994217928_0001_01_73 with incomplete > records > 2018-07-20 16:49:23,207 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container container_1531994217928_0001_01_74 with incomplete > records > 2018-07-20 16:49:23,207 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container container_1531994217928_0001_01_75 with incomplete > records > 2018-07-20 16:49:23,207 WARN > org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: > Remove container cont
[jira] [Commented] (YARN-8593) Add new RM web service endpoint to get cluster user info
[ https://issues.apache.org/jira/browse/YARN-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561457#comment-16561457 ] Akhil PB commented on YARN-8593: We need to show the requested user in UI client, which is given by *hsr.* As sunil mentioned, existing api does not take hsr. So we had to add new api which returns requested user and added rm login user to enrich the response. [~sunilg] [~rohithsharma] thoughts? > Add new RM web service endpoint to get cluster user info > > > Key: YARN-8593 > URL: https://issues.apache.org/jira/browse/YARN-8593 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8593.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8418) App local logs could leaked if log aggregation fails to initialize for the app
[ https://issues.apache.org/jira/browse/YARN-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561449#comment-16561449 ] Bibin A Chundatt commented on YARN-8418: Thank you [~leftnoteasy] for review # Removed the event based on appId, # Reduce the number of events added empty check based on invalid apps > App local logs could leaked if log aggregation fails to initialize for the app > -- > > Key: YARN-8418 > URL: https://issues.apache.org/jira/browse/YARN-8418 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.8.0, 3.0.0-alpha1 >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > Attachments: YARN-8418.001.patch, YARN-8418.002.patch, > YARN-8418.003.patch, YARN-8418.004.patch, YARN-8418.005.patch, > YARN-8418.006.patch, YARN-8418.007.patch, YARN-8418.008.patch > > > If log aggregation fails init createApp directory container logs could get > leaked in NM directory > For log running application restart of NM after token renewal this case is > possible/ Application submission with invalid delegation token -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8418) App local logs could leaked if log aggregation fails to initialize for the app
[ https://issues.apache.org/jira/browse/YARN-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bibin A Chundatt updated YARN-8418: --- Attachment: YARN-8418.008.patch > App local logs could leaked if log aggregation fails to initialize for the app > -- > > Key: YARN-8418 > URL: https://issues.apache.org/jira/browse/YARN-8418 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.8.0, 3.0.0-alpha1 >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > Attachments: YARN-8418.001.patch, YARN-8418.002.patch, > YARN-8418.003.patch, YARN-8418.004.patch, YARN-8418.005.patch, > YARN-8418.006.patch, YARN-8418.007.patch, YARN-8418.008.patch > > > If log aggregation fails init createApp directory container logs could get > leaked in NM directory > For log running application restart of NM after token renewal this case is > possible/ Application submission with invalid delegation token -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8602) make capacity-scheduler.xml file configurable in yarn-site?
Shen Yinjie created YARN-8602: - Summary: make capacity-scheduler.xml file configurable in yarn-site? Key: YARN-8602 URL: https://issues.apache.org/jira/browse/YARN-8602 Project: Hadoop YARN Issue Type: Improvement Reporter: Shen Yinjie Like Fair Scheduler? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7494) Add muti node lookup support for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561370#comment-16561370 ] Sunil Govindan commented on YARN-7494: -- [~leftnoteasy] [~cheersyang] pls help to review. > Add muti node lookup support for better placement > - > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch, > YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch, > YARN-7494.v0.patch, YARN-7494.v1.patch, multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8559) Expose mutable-conf scheduler's configuration in RM /scheduler-conf endpoint
[ https://issues.apache.org/jira/browse/YARN-8559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561338#comment-16561338 ] Weiwei Yang commented on YARN-8559: --- Hi [~suma.shivaprasad] Sorry for late response, I was on vacation. Thanks for reviewing this. May I know why we need a new endpoint for this? Giving "scheduler-conf" already exists, would it be confusing to add a new similar one? > Expose mutable-conf scheduler's configuration in RM /scheduler-conf endpoint > > > Key: YARN-8559 > URL: https://issues.apache.org/jira/browse/YARN-8559 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Reporter: Anna Savarin >Assignee: Weiwei Yang >Priority: Major > Attachments: YARN-8559.001.patch, YARN-8559.002.patch > > > All Hadoop services provide a set of common endpoints (/stacks, /logLevel, > /metrics, /jmx, /conf). In the case of the Resource Manager, part of the > configuration comes from the scheduler being used. Currently, these > configuration key/values are not exposed through the /conf endpoint, thereby > revealing an incomplete configuration picture. > Make an improvement and expose the scheduling configuration info through the > RM's /conf endpoint. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8418) App local logs could leaked if log aggregation fails to initialize for the app
[ https://issues.apache.org/jira/browse/YARN-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561188#comment-16561188 ] Wangda Tan commented on YARN-8418: -- Thanks [~bibinchundatt] for updating the patch and [~suma.shivaprasad] for code review. One minor comments: ContainerManagerImpl#handleCredentialUpdate, instead of calling following logic: {code} Set invalidTokenApps = logHandler.getInvalidTokenApps(); for (ApplicationId app : invalidTokenApps) { if (context.getSystemCredentialsForApps().get(app) != null) { dispatcher.getEventHandler() .handle(new LogHandlerTokenUpdatedEvent(app)); } } {code} Is it better to just send a LogHandlerTokenUpdatedEvent to log handler (but no need to specify app id). And inside LogHandler, loop the invalidTokenApps, and update token. Benefits of doing this: - ContainerManagerImpl doesn't have to know about invalid apps, better encapsulation. - Avoid race condition that app get added to invalidApps while looping apps from handleCredentialUpdate. > App local logs could leaked if log aggregation fails to initialize for the app > -- > > Key: YARN-8418 > URL: https://issues.apache.org/jira/browse/YARN-8418 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.8.0, 3.0.0-alpha1 >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > Attachments: YARN-8418.001.patch, YARN-8418.002.patch, > YARN-8418.003.patch, YARN-8418.004.patch, YARN-8418.005.patch, > YARN-8418.006.patch, YARN-8418.007.patch > > > If log aggregation fails init createApp directory container logs could get > leaked in NM directory > For log running application restart of NM after token renewal this case is > possible/ Application submission with invalid delegation token -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8175) Add support for Node Labels in SLS
[ https://issues.apache.org/jira/browse/YARN-8175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561184#comment-16561184 ] Íñigo Goiri commented on YARN-8175: --- Is the {{TestAMRMProxy}} error related? I don't think so as the logic doesn't change but can you double check? > Add support for Node Labels in SLS > -- > > Key: YARN-8175 > URL: https://issues.apache.org/jira/browse/YARN-8175 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Abhishek Modi >Assignee: Abhishek Modi >Priority: Major > Attachments: YARN-8175.001.patch, YARN-8175.002.patch, > YARN-8175.003.patch, YARN-8175.004.patch, YARN-8175.005.patch, > YARN-8175.006.patch, YARN-8175.007.patch, YARN-8175.008.patch > > > Currently, SLS doesn't support node labels. With this jira, we are planning > to add support for node labels in SLS. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7863) Modify placement constraints to support node attributes
[ https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561134#comment-16561134 ] Naganarasimha G R commented on YARN-7863: - Hi [~sunilg], Patch seems to fail after rebase (tried locally too). Can you please take a look at it ? > Modify placement constraints to support node attributes > --- > > Key: YARN-7863 > URL: https://issues.apache.org/jira/browse/YARN-7863 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7863-YARN-3409.002.patch, > YARN-7863-YARN-3409.003.patch, YARN-7863.v0.patch > > > This Jira will track to *Modify existing placement constraints to support > node attributes.* -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7863) Modify placement constraints to support node attributes
[ https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R reassigned YARN-7863: --- Assignee: Sunil Govindan (was: Naganarasimha G R) > Modify placement constraints to support node attributes > --- > > Key: YARN-7863 > URL: https://issues.apache.org/jira/browse/YARN-7863 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7863-YARN-3409.002.patch, > YARN-7863-YARN-3409.003.patch, YARN-7863.v0.patch > > > This Jira will track to *Modify existing placement constraints to support > node attributes.* -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7863) Modify placement constraints to support node attributes
[ https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R reassigned YARN-7863: --- Assignee: Naganarasimha G R (was: Sunil Govindan) > Modify placement constraints to support node attributes > --- > > Key: YARN-7863 > URL: https://issues.apache.org/jira/browse/YARN-7863 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Assignee: Naganarasimha G R >Priority: Major > Attachments: YARN-7863-YARN-3409.002.patch, > YARN-7863-YARN-3409.003.patch, YARN-7863.v0.patch > > > This Jira will track to *Modify existing placement constraints to support > node attributes.* -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8175) Add support for Node Labels in SLS
[ https://issues.apache.org/jira/browse/YARN-8175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561076#comment-16561076 ] genericqa commented on YARN-8175: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 26s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 4 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 49s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m 51s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 29m 8s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 15s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 13m 0s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 30s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 27m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 13s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 6s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 45s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 23m 33s{color} | {color:red} hadoop-yarn-client in the patch failed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m 24s{color} | {color:green} hadoop-sls in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 42s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}154m 51s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.client.api.impl.TestAMRMProxy | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 | | JIRA Issue | YARN-8175 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12933507/YARN-8175.008.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 83dd8b65fc53 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 007e6f5 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_171 | | findbugs | v3.1.0-RC1 | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/21425/artifact/out/
[jira] [Updated] (YARN-8175) Add support for Node Labels in SLS
[ https://issues.apache.org/jira/browse/YARN-8175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Modi updated YARN-8175: Attachment: YARN-8175.008.patch > Add support for Node Labels in SLS > -- > > Key: YARN-8175 > URL: https://issues.apache.org/jira/browse/YARN-8175 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Abhishek Modi >Assignee: Abhishek Modi >Priority: Major > Attachments: YARN-8175.001.patch, YARN-8175.002.patch, > YARN-8175.003.patch, YARN-8175.004.patch, YARN-8175.005.patch, > YARN-8175.006.patch, YARN-8175.007.patch, YARN-8175.008.patch > > > Currently, SLS doesn't support node labels. With this jira, we are planning > to add support for node labels in SLS. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org