Re: [ANNOUNCE] Eric Badger is now a committer!

Congrats Eric ! On Thu, Mar 7, 2019 at 5:06 AM zhankun tang wrote: > Congratulations! > > Zhankun > > 俊平堵 于2019年3月7日 周四下午12:34写道： > > > Congrats, Eric! > > > > Thanks, > > > > Junping > > > > Eric Payne 于2019年3月6日周三 上午1:20写道： > > > > > It is my pleasure to announce that Eric Badger has

Re: [VOTE] Release Apache Hadoop 3.1.2 - RC1

+1 - Built from source - Deployed a single node cluster - Ran few MR and Distributed Shell Jobs - Check YARN UI2 flow with Yarn services and jobs Thanks Suma On Mon, Feb 4, 2019 at 7:51 PM Wilfred Spiegelenburg wrote: > +1 (non binding) > > - build from source on MacOSX 10.14.2, 1.8.0u181 > -

Re: [VOTE] Propose to start new Hadoop sub project "submarine"

+1 ( non binding) On Mon, Feb 4, 2019 at 7:23 AM Shashikant Banerjee < sbaner...@hortonworks.com> wrote: > +1 (non-binding) > > Thanks > Shashi > > On 2/4/19, 8:27 PM, "Elek, Marton" wrote: > > +1 (non-binding) > > (my arguments are in the discuss thread. small move, huge benefit) > >

Re: [DISCUSS] Making submarine to different release model like Ozone

+1. Thanks for bringing this up Wangda. Makes sense to have Submarine follow its own release cadence given the good momentum/adoption so far. Also, making it run with older versions of Hadoop would drive higher adoption. Suma On Fri, Feb 1, 2019 at 9:40 AM Eric Yang wrote: > Submarine is an

[jira] [Created] (YARN-9039) App ACLs are not validated when serving logs from Logs CLI/Yarn UI2

Suma Shivaprasad created YARN-9039: -- Summary: App ACLs are not validated when serving logs from Logs CLI/Yarn UI2 Key: YARN-9039 URL: https://issues.apache.org/jira/browse/YARN-9039 Project: Hadoop

[jira] [Created] (YARN-9030) Log aggregation changes to handle filesystems which do not support permissions

Suma Shivaprasad created YARN-9030: -- Summary: Log aggregation changes to handle filesystems which do not support permissions Key: YARN-9030 URL: https://issues.apache.org/jira/browse/YARN-9030

[jira] [Resolved] (YARN-4249) Many options in "yarn application" command is not documented

[ https://issues.apache.org/jira/browse/YARN-4249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad resolved YARN-4249. Resolution: Not A Problem > Many options in "yarn application" command is n

[jira] [Created] (YARN-8923) Cleanup references to ENV file type in code

Suma Shivaprasad created YARN-8923: -- Summary: Cleanup references to ENV file type in code Key: YARN-8923 URL: https://issues.apache.org/jira/browse/YARN-8923 Project: Hadoop YARN Issue

[jira] [Created] (YARN-8920) LogAggregation should be configurable to allow writing to underlying storage as appOwner or yarn user

Suma Shivaprasad created YARN-8920: -- Summary: LogAggregation should be configurable to allow writing to underlying storage as appOwner or yarn user Key: YARN-8920 URL: https://issues.apache.org/jira/browse/YARN

[jira] [Created] (YARN-8711) YARN UI2 : Display component state in Component list and details page for a Service

Suma Shivaprasad created YARN-8711: -- Summary: YARN UI2 : Display component state in Component list and details page for a Service Key: YARN-8711 URL: https://issues.apache.org/jira/browse/YARN-8711

[jira] [Created] (YARN-8710) Service AM should set a finite limit on NM container max retries

Suma Shivaprasad created YARN-8710: -- Summary: Service AM should set a finite limit on NM container max retries Key: YARN-8710 URL: https://issues.apache.org/jira/browse/YARN-8710 Project: Hadoop

[jira] [Created] (YARN-8675) Setting hostname of docker container breaks with "host" networking mode for Apps which do not run as a YARN service

Suma Shivaprasad created YARN-8675: -- Summary: Setting hostname of docker container breaks with "host" networking mode for Apps which do not run as a YARN service Key: YARN-8675 URL: https://issues.

Re: [VOTE] Release Apache Hadoop 3.1.1 - RC0

Thanks Wangda! +1 (non-binding) Tested the following: - Built from source - Setup single node cluster - Tested Dynamic queues - Tested MR and DS with default, docker runtime - Tested Yarn Services with various restart policies Thanks Suma On Tue, Aug 7, 2018 at 2:45 PM Eric Payne

[jira] [Created] (YARN-8588) Logging improvements for better debuggability

Suma Shivaprasad created YARN-8588: -- Summary: Logging improvements for better debuggability Key: YARN-8588 URL: https://issues.apache.org/jira/browse/YARN-8588 Project: Hadoop YARN Issue

[jira] [Created] (YARN-8572) YarnClient getContainers API should support filtering by container status

Suma Shivaprasad created YARN-8572: -- Summary: YarnClient getContainers API should support filtering by container status Key: YARN-8572 URL: https://issues.apache.org/jira/browse/YARN-8572 Project

[jira] [Created] (YARN-8519) Yarn UI2 : Changes to depict Auto Created leaf Queues/Managed Queues differently from other queues

Suma Shivaprasad created YARN-8519: -- Summary: Yarn UI2 : Changes to depict Auto Created leaf Queues/Managed Queues differently from other queues Key: YARN-8519 URL: https://issues.apache.org/jira/browse/YARN

[jira] [Created] (YARN-8419) In "New Service" section of new YARN UI, user cannot submit service as Submit button is always disabled.

Suma Shivaprasad created YARN-8419: -- Summary: In "New Service" section of new YARN UI, user cannot submit service as Submit button is always disabled. Key: YARN-8419 URL: https://issues.apache.org/j

[jira] [Created] (YARN-8415) TimelineWebServices should throw a ForBiddenException ( 403

Suma Shivaprasad created YARN-8415: -- Summary: TimelineWebServices should throw a ForBiddenException ( 403 Key: YARN-8415 URL: https://issues.apache.org/jira/browse/YARN-8415 Project: Hadoop YARN

[jira] [Created] (YARN-8339) Service AM should localize static/archive resource types to container working directory instead of 'resources' diurectory

Suma Shivaprasad created YARN-8339: -- Summary: Service AM should localize static/archive resource types to container working directory instead of 'resources' diurectory Key: YARN-8339 URL: https

[jira] [Created] (YARN-8300) Fix NPE in DefaultUpgradeComponentsFinder

Suma Shivaprasad created YARN-8300: -- Summary: Fix NPE in DefaultUpgradeComponentsFinder Key: YARN-8300 URL: https://issues.apache.org/jira/browse/YARN-8300 Project: Hadoop YARN Issue Type

[jira] [Created] (YARN-8296) Update YarnServiceApi documentation and yarn service UI code to remove references to unique_component_support

Suma Shivaprasad created YARN-8296: -- Summary: Update YarnServiceApi documentation and yarn service UI code to remove references to unique_component_support Key: YARN-8296 URL: https://issues.apache.org/jira

[jira] [Created] (YARN-8255) Allow option to disable flex for a service component

Suma Shivaprasad created YARN-8255: -- Summary: Allow option to disable flex for a service component Key: YARN-8255 URL: https://issues.apache.org/jira/browse/YARN-8255 Project: Hadoop YARN

[jira] [Created] (YARN-8217) RmAuthenticationFilterInitializer /TimelineAuthenticationFilterInitializer should use Configuration.getPropsWithPrefix instead of iterator

Suma Shivaprasad created YARN-8217: -- Summary: RmAuthenticationFilterInitializer /TimelineAuthenticationFilterInitializer should use Configuration.getPropsWithPrefix instead of iterator Key: YARN-8217 URL: https

[jira] [Created] (YARN-8210) AMRMClient logging on every heartbeat to track updation of AM RM token causes too many log lines to be generated in AM logs

Suma Shivaprasad created YARN-8210: -- Summary: AMRMClient logging on every heartbeat to track updation of AM RM token causes too many log lines to be generated in AM logs Key: YARN-8210 URL: https

[jira] [Created] (YARN-8177) Fix documentation for node label support

Suma Shivaprasad created YARN-8177: -- Summary: Fix documentation for node label support Key: YARN-8177 URL: https://issues.apache.org/jira/browse/YARN-8177 Project: Hadoop YARN Issue Type

[jira] [Created] (YARN-8121) Container runtime and runtime context information should be available in UI

Suma Shivaprasad created YARN-8121: -- Summary: Container runtime and runtime context information should be available in UI Key: YARN-8121 URL: https://issues.apache.org/jira/browse/YARN-8121 Project

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

+1 (non binding) *Verified - User Group Queue mapping - Node labels with New UI- Dynamic queuesThanksSuma* On Wed, Apr 4, 2018 at 11:48 AM, Steve Loughran wrote: > that's "dangerously interesting". I think you are right, and I also think > it'll just be the version

[jira] [Created] (YARN-8086) ManagedParentQueue with no leaf queues cause JS error in new UI

Suma Shivaprasad created YARN-8086: -- Summary: ManagedParentQueue with no leaf queues cause JS error in new UI Key: YARN-8086 URL: https://issues.apache.org/jira/browse/YARN-8086 Project: Hadoop YARN

[jira] [Resolved] (YARN-7987) Docker container name(--name) needs to be DNS friendly for DNS resolution to work in user defined networks.

[ https://issues.apache.org/jira/browse/YARN-7987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad resolved YARN-7987. Resolution: Won't Fix > Docker container name(--name) needs to be DNS friendly for

[jira] [Created] (YARN-7994) Add support for network-alias in docker run for user defined networks

Suma Shivaprasad created YARN-7994: -- Summary: Add support for network-alias in docker run for user defined networks Key: YARN-7994 URL: https://issues.apache.org/jira/browse/YARN-7994 Project

[jira] [Created] (YARN-7987) Docker container name(--name) needs to be DNS friendly for DNS resolution to work in user defined networks.

Suma Shivaprasad created YARN-7987: -- Summary: Docker container name(--name) needs to be DNS friendly for DNS resolution to work in user defined networks. Key: YARN-7987 URL: https://issues.apache.org/jira

[jira] [Created] (YARN-7935) Expose container's hostname to applications running within the docker container

Suma Shivaprasad created YARN-7935: -- Summary: Expose container's hostname to applications running within the docker container Key: YARN-7935 URL: https://issues.apache.org/jira/browse/YARN-7935

[jira] [Created] (YARN-7657) Queue Mapping could provide options to provide 'user' specific auto-created queues under a specified group parent queue

Suma Shivaprasad created YARN-7657: -- Summary: Queue Mapping could provide options to provide 'user' specific auto-created queues under a specified group parent queue Key: YARN-7657 URL: https://issues.apache.org

[jira] [Created] (YARN-7643) Handle recovery of applications on auto-created leaf queues

Suma Shivaprasad created YARN-7643: -- Summary: Handle recovery of applications on auto-created leaf queues Key: YARN-7643 URL: https://issues.apache.org/jira/browse/YARN-7643 Project: Hadoop YARN

[jira] [Created] (YARN-7639) Queue Management scheduling edit policy class needs to be configured dynamically

Suma Shivaprasad created YARN-7639: -- Summary: Queue Management scheduling edit policy class needs to be configured dynamically Key: YARN-7639 URL: https://issues.apache.org/jira/browse/YARN-7639

[jira] [Created] (YARN-7638) Add unit tests for Preemption and Recovery

Suma Shivaprasad created YARN-7638: -- Summary: Add unit tests for Preemption and Recovery Key: YARN-7638 URL: https://issues.apache.org/jira/browse/YARN-7638 Project: Hadoop YARN Issue Type

[jira] [Created] (YARN-7634) Queue ACL validations should validate parent queue ACLs before auto-creating leaf queues

Suma Shivaprasad created YARN-7634: -- Summary: Queue ACL validations should validate parent queue ACLs before auto-creating leaf queues Key: YARN-7634 URL: https://issues.apache.org/jira/browse/YARN-7634

[jira] [Created] (YARN-7633) Add documentation for auto queue creation feature and related configurations

Suma Shivaprasad created YARN-7633: -- Summary: Add documentation for auto queue creation feature and related configurations Key: YARN-7633 URL: https://issues.apache.org/jira/browse/YARN-7633 Project

[jira] [Created] (YARN-7632) Effective min and max resource need to be set for auto created leaf queues upon creation and capacity management

Suma Shivaprasad created YARN-7632: -- Summary: Effective min and max resource need to be set for auto created leaf queues upon creation and capacity management Key: YARN-7632 URL: https://issues.apache.org/jira

[jira] [Created] (YARN-7586) Application Placement should be done before ACL checks in ResourceManager

Suma Shivaprasad created YARN-7586: -- Summary: Application Placement should be done before ACL checks in ResourceManager Key: YARN-7586 URL: https://issues.apache.org/jira/browse/YARN-7586 Project

[jira] [Created] (YARN-7574) Add support for Node Labels on Auto Created Leaf Queue Template

Suma Shivaprasad created YARN-7574: -- Summary: Add support for Node Labels on Auto Created Leaf Queue Template Key: YARN-7574 URL: https://issues.apache.org/jira/browse/YARN-7574 Project: Hadoop YARN

[jira] [Created] (YARN-7520) Queue Ordering policy changes for ordering auto created leaf queues within Managed parent Queues

Suma Shivaprasad created YARN-7520: -- Summary: Queue Ordering policy changes for ordering auto created leaf queues within Managed parent Queues Key: YARN-7520 URL: https://issues.apache.org/jira/browse/YARN-7520

[jira] [Created] (YARN-7473) Implement Framework and policy for capacity management of auto created queues

Suma Shivaprasad created YARN-7473: -- Summary: Implement Framework and policy for capacity management of auto created queues Key: YARN-7473 URL: https://issues.apache.org/jira/browse/YARN-7473

[jira] [Created] (YARN-7420) YARN UI changes to depict auto created queues

Suma Shivaprasad created YARN-7420: -- Summary: YARN UI changes to depict auto created queues Key: YARN-7420 URL: https://issues.apache.org/jira/browse/YARN-7420 Project: Hadoop YARN Issue

[jira] [Created] (YARN-7419) Implement Auto Queue Creation with modifications to queue mapping flow

Suma Shivaprasad created YARN-7419: -- Summary: Implement Auto Queue Creation with modifications to queue mapping flow Key: YARN-7419 URL: https://issues.apache.org/jira/browse/YARN-7419 Project

[jira] [Created] (YARN-7394) Merge code paths for Reservation/Plan queues and Auto Created queues

Suma Shivaprasad created YARN-7394: -- Summary: Merge code paths for Reservation/Plan queues and Auto Created queues Key: YARN-7394 URL: https://issues.apache.org/jira/browse/YARN-7394 Project: Hadoop

[jira] [Created] (YARN-7103) Add logs for errors while signalling containers for better debuggability

Suma Shivaprasad created YARN-7103: -- Summary: Add logs for errors while signalling containers for better debuggability Key: YARN-7103 URL: https://issues.apache.org/jira/browse/YARN-7103 Project

Deadlock in RM

We are observing a repetitive issue with hadoop 2.6.0/HDP 2.2 with RM capacity scheduler threads getting into a deadlock. It appears that the RM UI scheduler flow for getResourceUsageReport is triggering the deadlock. Have raised a jira for the same -

[jira] [Created] (YARN-3346) Deadlock in Capacity Scheduler

Suma Shivaprasad created YARN-3346: -- Summary: Deadlock in Capacity Scheduler Key: YARN-3346 URL: https://issues.apache.org/jira/browse/YARN-3346 Project: Hadoop YARN Issue Type: Bug

Re: FileSystem Vs ZKStateStore for RM recovery

ResourceManagers with failover. ZK-store ensures a single RM has write access and thus is better protected against split-brain cases where both RMs think they are active. On Tue, Feb 10, 2015 at 9:59 PM, Suma Shivaprasad sumasai.shivapra...@gmail.com wrote: We are planning to deploy Hadoop 2.6.0

Re: QueueMetrics.AppsKilled/Failed metrics and failure reasons

-Original Message- From: Suma Shivaprasad [mailto:sumasai.shivapra...@gmail.com] Sent: 04 February 2015 11:03 To: u...@hadoop.apache.org Cc: yarn-dev@hadoop.apache.org Subject: Re: QueueMetrics.AppsKilled/Failed metrics and failure reasons Using hadoop 2.4.0. #of Applications running

Re: QueueMetrics.AppsKilled/Failed metrics and failure reasons

is you can check in NodeManager logs also. Here you need to check using container_id for corresponding application. Thanks Regards Rohith Sharma K S *From:* Suma Shivaprasad [mailto:sumasai.shivapra...@gmail.com] *Sent:* 03 February 2015 21:35 *To:* u...@hadoop.apache.org; yarn-dev

QueueMetrics.AppsKilled/Failed metrics and failure reasons

Hello, Was trying to debug reasons for Killed/Failed apps and was checking for the applications that were killed/failed in RM logs - from RMAuditLogger. QueueMetrics.AppsKilled/Failed metrics shows much higher nos i.e ~100. However RMAuditLogger shows 1 or 2 Apps as Killed/Failed in the logs. Is