[jira] [Resolved] (YARN-7031) Support distributed node attributes
[ https://issues.apache.org/jira/browse/YARN-7031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bibin A Chundatt resolved YARN-7031. Resolution: Implemented Fix Version/s: YARN-3409 Closing this since its already handled > Support distributed node attributes > --- > > Key: YARN-7031 > URL: https://issues.apache.org/jira/browse/YARN-7031 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Major > Fix For: YARN-3409 > > Attachments: Distributed node attributes v1.pdf, > YARN-7031-YARN-3409.001.patch > > > Allow nodemanagers to push its attributes to RM -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8517) getContainer and getContainers ResourceManager REST API methods are not documented
Szilard Nemeth created YARN-8517:
Summary: getContainer and getContainers ResourceManager REST API
methods are not documented
Key: YARN-8517
URL: https://issues.apache.org/jira/browse/YARN-8517
Project: Hadoop YARN
Issue Type: Improvement
Components: resourcemanager
Reporter: Szilard Nemeth
Looking at the documentation here:
https://hadoop.apache.org/docs/r3.1.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html
I cannot find documentation for 2 RM REST endpoints:
- /apps/{appid}/appattempts/{appattemptid}/containers
- /apps/{appid}/appattempts/{appattemptid}/containers/{containerid}
I suppose they are not intentionally undocumented.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8518) test-container-executor test_is_empty() is broken
Jim Brennan created YARN-8518: - Summary: test-container-executor test_is_empty() is broken Key: YARN-8518 URL: https://issues.apache.org/jira/browse/YARN-8518 Project: Hadoop YARN Issue Type: Bug Reporter: Jim Brennan A new test was recently added to test-container-executor.c that has some problems. It is attempting to mkdir() a hard-coded path: /tmp/2938rf2983hcqnw8ud/emptydir This fails because the base directory is not there. These directories are not being cleaned up either. It should be using TEST_ROOT. I don't know what Jira this change was made under - the git commit from July 9 2018 does not reference a Jira. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8519) Yarn UI2 : Changes to depict Auto Created leaf Queues/Managed Queues differently from other queues
Suma Shivaprasad created YARN-8519: -- Summary: Yarn UI2 : Changes to depict Auto Created leaf Queues/Managed Queues differently from other queues Key: YARN-8519 URL: https://issues.apache.org/jira/browse/YARN-8519 Project: Hadoop YARN Issue Type: Sub-task Reporter: Suma Shivaprasad YARN-7420 covers changes to depict auto created leaf queues in a separate color notation but this was done in the old Yarn UI and similiar chnages need to be incorporated in the new YARN UI to depict Managed Parent queues/Auto-Created leaf queues separately -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Resolved] (YARN-4192) Add YARN metric logging periodically to a seperate file
[ https://issues.apache.org/jira/browse/YARN-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YARN-4192. --- Resolution: Won't Fix I am closing this now, like [~aw] mentioned, this should be already handled by metrics2. Please reopen it if you think otherwise [~nijel]. Thanks. > Add YARN metric logging periodically to a seperate file > --- > > Key: YARN-4192 > URL: https://issues.apache.org/jira/browse/YARN-4192 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: nijel >Assignee: nijel >Priority: Minor > > HDFS-8880 added a framework for logging metrics in a given interval. > This can be added to YARN as well > Any thoughts ? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Resolved] (YARN-8241) MRAppMaster fails when using UID:GID pair within docker container
[ https://issues.apache.org/jira/browse/YARN-8241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Badger resolved YARN-8241. --- Resolution: Won't Fix We do not plan to support running containers in which a user lookup cannot be performed. This is to be considered a base requirement for a container to run. Some sort of user lookup mechanism such as nscd, sssd, /etc/passwd, or something else needs to be implemented so that the UID:GID pair can be linked to a user and group > MRAppMaster fails when using UID:GID pair within docker container > - > > Key: YARN-8241 > URL: https://issues.apache.org/jira/browse/YARN-8241 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Badger >Priority: Major > Labels: Docker > > As mentioned in [this > comment|https://issues.apache.org/jira/browse/YARN-4266?focusedCommentId=16063931=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16063931], > the MRAppMaster fails for docker containers if there is no additional user > lookup strategy (e.g. bind-mounting /var/run/nscd or /etc/passwd). We need a > better solution so that users can still run even if they are not known inside > of the container by name -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Resolved] (YARN-6576) Improve Diagonstic by moving Error stack trace from NM to slider AM
[ https://issues.apache.org/jira/browse/YARN-6576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang resolved YARN-6576. - Resolution: Duplicate Assignee: Eric Yang This is duplicate of YARN-8403. > Improve Diagonstic by moving Error stack trace from NM to slider AM > --- > > Key: YARN-6576 > URL: https://issues.apache.org/jira/browse/YARN-6576 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Yesha Vora >Assignee: Eric Yang >Priority: Major > Labels: Docker > > Slider Master diagonstics should improve to show root cause of App failures > for issues like missing docker image. > Currently, Slider Master log does not show proper error message to debug such > failure. User have to access Nodemanager logs to find out root cause of such > issues where container failed to start. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Resolved] (YARN-7197) Add support for a volume blacklist for docker containers
[ https://issues.apache.org/jira/browse/YARN-7197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Badger resolved YARN-7197. --- Resolution: Won't Fix There are currently no plans to attempt to implement this feature. Feel free to re-open if you would like to work on this > Add support for a volume blacklist for docker containers > > > Key: YARN-7197 > URL: https://issues.apache.org/jira/browse/YARN-7197 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Shane Kumpf >Priority: Major > Labels: Docker > Attachments: YARN-7197.001.patch, YARN-7197.002.patch, > YARN-7197.003.patch, YARN-7197.004.patch, YARN-7197.005.patch > > > Docker supports bind mounting host directories into containers. Work is > underway to allow admins to configure a whilelist of volume mounts. While > this is a much needed and useful feature, it opens the door for > misconfiguration that may lead to users being able to compromise or crash the > system. > One example would be allowing users to mount /run from a host running > systemd, and then running systemd in that container, rendering the host > mostly unusable. > This issue is to add support for a default blacklist. The default blacklist > would be where we put files and directories that if mounted into a container, > are likely to have negative consequences. Users are encouraged not to remove > items from the default blacklist, but may do so if necessary. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8520) Document best practice for user management
Eric Yang created YARN-8520: --- Summary: Document best practice for user management Key: YARN-8520 URL: https://issues.apache.org/jira/browse/YARN-8520 Project: Hadoop YARN Issue Type: Sub-task Components: documentation, yarn Reporter: Eric Yang Assignee: Eric Yang Docker container must have consistent username and groups with host operating system when external mount points are exposed to docker container. This prevents malicious or unauthorized impersonation to occur. This task is to document the best practice to ensure user and group membership are consistent across docker containers. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Resolved] (YARN-3611) Support Docker Containers In LinuxContainerExecutor
[ https://issues.apache.org/jira/browse/YARN-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Badger resolved YARN-3611. --- Resolution: Fixed We have closed out Phase 1 of Docker container support and are now moving onto Phase 2. Phase 2 will be tracked by YARN-8472, so please file any new JIRAs under that umbrella (with the 'Docker' label). > Support Docker Containers In LinuxContainerExecutor > --- > > Key: YARN-3611 > URL: https://issues.apache.org/jira/browse/YARN-3611 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Reporter: Sidharta Seethana >Assignee: Sidharta Seethana >Priority: Major > Labels: Docker > > Support Docker Containers In LinuxContainerExecutor > LinuxContainerExecutor provides useful functionality today with respect to > localization, cgroups based resource management and isolation for CPU, > network, disk etc. as well as security with a well-defined mechanism to > execute privileged operations using the container-executor utility. Bringing > docker support to LinuxContainerExecutor lets us use all of this > functionality when running docker containers under YARN, while not requiring > users and admins to configure and use a different ContainerExecutor. > There are several aspects here that need to be worked through : > * Mechanism(s) to let clients request docker-specific functionality - we > could initially implement this via environment variables without impacting > the client API. > * Security - both docker daemon as well as application > * Docker image localization > * Running a docker container via container-executor as a specified user > * “Isolate” the docker container in terms of CPU/network/disk/etc > * Communicating with and/or signaling the running container (ensure correct > pid handling) > * Figure out workarounds for certain performance-sensitive scenarios like > HDFS short-circuit reads > * All of these need to be achieved without changing the current behavior of > LinuxContainerExecutor -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
