[jira] [Created] (YARN-8897) LoadBasedRouterPolicy throws "NullPointerException" if no sub cluster is available with cluster weight=1

2018-10-16 Thread Akshay Agarwal (JIRA)
Akshay Agarwal created YARN-8897:


 Summary: LoadBasedRouterPolicy throws "NullPointerException" if no 
sub cluster is available with cluster weight=1 
 Key: YARN-8897
 URL: https://issues.apache.org/jira/browse/YARN-8897
 Project: Hadoop YARN
  Issue Type: Bug
  Components: federation, router
Reporter: Akshay Agarwal


If no sub clusters are available for "*Load Based Router Policy*" with *cluster 
weight* as *1*  in Router Based Federation Setup , throwing 
"*NullPointerException*".

 

*Exception Details:*
{code:java}
java.lang.NullPointerException: java.lang.NullPointerException
 at 
org.apache.hadoop.yarn.server.federation.policies.router.LoadBasedRouterPolicy.getHomeSubcluster(LoadBasedRouterPolicy.java:99)
 at 
org.apache.hadoop.yarn.server.federation.policies.RouterPolicyFacade.getHomeSubcluster(RouterPolicyFacade.java:204)
 at 
org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.submitApplication(FederationClientInterceptor.java:362)
 at 
org.apache.hadoop.yarn.server.router.clientrm.RouterClientRMService.submitApplication(RouterClientRMService.java:218)
 at 
org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitApplication(ApplicationClientProtocolPBServiceImpl.java:282)
 at 
org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:579)
 at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
 at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
 at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:872)
 at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:818)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
 at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2678)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
 at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
 at org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53)
 at 
org.apache.hadoop.yarn.ipc.RPCUtil.instantiateRuntimeException(RPCUtil.java:85)
 at org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:122)
 at 
org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.submitApplication(ApplicationClientProtocolPBClientImpl.java:297)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
 at 
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
 at 
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
 at 
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
 at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
 at com.sun.proxy.$Proxy15.submitApplication(Unknown Source)
 at 
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:288)
 at 
org.apache.hadoop.mapred.ResourceMgrDelegate.submitApplication(ResourceMgrDelegate.java:300)
 at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:331)
 at 
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:254)
 at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1570)
 at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1567)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
 at org.apache.hadoop.mapreduce.Job.submit(Job.java:1567)
 at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1588)
 at 
org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:307)
 at org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:360)
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
 at org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:368)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(Delegatin

[jira] [Created] (YARN-8896) Limit the maximum number of container assignments per heartbeat

2018-10-16 Thread Weiwei Yang (JIRA)
Weiwei Yang created YARN-8896:
-

 Summary: Limit the maximum number of container assignments per 
heartbeat
 Key: YARN-8896
 URL: https://issues.apache.org/jira/browse/YARN-8896
 Project: Hadoop YARN
  Issue Type: Improvement
Affects Versions: 3.0.0, 2.9.0
Reporter: Weiwei Yang


YARN-4161 adds a configuration 
\{{yarn.scheduler.capacity.per-node-heartbeat.maximum-container-assignments}} 
to control max number of container assignments per heartbeat, however the 
default value is -1. This could potentially cause the CS gets stuck in the 
while loop causing issue like YARN-8513. We should change this to a finite 
number, e.g 100.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org



[jira] [Created] (YARN-8895) Improve YARN

2018-10-16 Thread Young Chen (JIRA)
Young Chen created YARN-8895:


 Summary: Improve YARN 
 Key: YARN-8895
 URL: https://issues.apache.org/jira/browse/YARN-8895
 Project: Hadoop YARN
  Issue Type: Improvement
Reporter: Young Chen
Assignee: Young Chen


Currently identifying error sources can be quite difficult, as they are written 
into an unstructured string "diagnostics" field. This is present in container 
statuses returned to the RM and in application attempts in the RM. These errors 
are difficult to classify without hard-coding diagnostic string searches.

This Jira aims to add a structured error field in NM and RM that preserves 
failure information and source component to enable faster and clearer error 
diagnosis.

Old error:

E.g.: 
Application application_1539325316309_0001 failed 1 times due to AM Container 
for appattempt_1539325316309_0001_01 exited with exitCode: 57005
For more detailed output, check application tracking 
page:http://:80/cluster/app/application_1539325316309_0001Then, click 
on links to logs of each attempt.
Diagnostics: Container exited with a non-zero exit code 57005
Failing this attempt. Failing the application.
 
Proposed new error example:
{code:java}
{"errors":[{"errorId":"E_SYSTEM_AM_AMCRASHED",
"name":"AM_CRASHED","severity":"Error",
"component":"AM",
"source":"System",
"exitType":"CONTAINER_FINISHED","containerStatus":57005,
"description":"Application attempt appattempt_1539325316309_0001_01 
encountered an error",
"helpLink":"http://XXX:80/proxy/application_1539325316309_0001/"}]}
{code}
 
 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org



[jira] [Created] (YARN-8894) Improve InMemoryPlan.java toString

2018-10-16 Thread BELUGA BEHR (JIRA)
BELUGA BEHR created YARN-8894:
-

 Summary: Improve InMemoryPlan.java toString
 Key: YARN-8894
 URL: https://issues.apache.org/jira/browse/YARN-8894
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: reservation system
Affects Versions: 3.2.0
Reporter: BELUGA BEHR


* Replace {{StringBuffer}} with {{StringBuilder}}
* Add spaces between fields for readability



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org



[jira] [Created] (YARN-8893) [AMRMProxy] Fix thread leak in AMRMClientRelayer and UAM client

2018-10-16 Thread Botong Huang (JIRA)
Botong Huang created YARN-8893:
--

 Summary: [AMRMProxy] Fix thread leak in AMRMClientRelayer and UAM 
client
 Key: YARN-8893
 URL: https://issues.apache.org/jira/browse/YARN-8893
 Project: Hadoop YARN
  Issue Type: Task
Reporter: Botong Huang
Assignee: Botong Huang


Fix thread leak in AMRMClientRelayer and UAM client used by 
FederationInterceptor, when destroying the interceptor pipeline in AMRMProxy. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org



[jira] [Created] (YARN-8892) YARN UI2 doc improvement to update security status

2018-10-16 Thread Sunil Govindan (JIRA)
Sunil Govindan created YARN-8892:


 Summary: YARN UI2 doc improvement to update security status
 Key: YARN-8892
 URL: https://issues.apache.org/jira/browse/YARN-8892
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Sunil Govindan
Assignee: Sunil Govindan
 Fix For: 3.2.0


UI2 is now tested under kerberized env as well. update this in the doc



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org



Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2018-10-16 Thread Apache Jenkins Server
For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/928/

[Oct 15, 2018 10:08:42 AM] (sunilg) YARN-8836. Add tags and attributes in 
resource definition. Contributed
[Oct 15, 2018 10:18:26 AM] (vinayakumarb) Fix potential FSImage corruption. 
Contributed by Daryn Sharp.
[Oct 15, 2018 10:40:25 AM] (bibinchundatt) YARN-8830. SLS tool fix node 
addition. Contributed by Bibin A Chundatt.
[Oct 15, 2018 3:51:57 PM] (sunilg) YARN-8869. YARN Service Client might not 
work correctly with RM REST API
[Oct 15, 2018 4:37:20 PM] (haibochen) YARN-8775. 
TestDiskFailures.testLocalDirsFailures sometimes can fail on
[Oct 15, 2018 4:51:26 PM] (inigoiri) HDFS-13987. RBF: Review of RandomResolver 
Class. Contributed by BELUGA
[Oct 15, 2018 5:51:55 PM] (xiao) HADOOP-14445. Addendum: Use 
DelegationTokenIssuer to create KMS
[Oct 15, 2018 6:52:38 PM] (jitendra) HDDS-629. Make ApplyTransaction calls in 
ContainerStateMachine
[Oct 15, 2018 9:53:55 PM] (stevel) HADOOP-15851. Disable wildfly logs to the 
console. Contributed by
[Oct 15, 2018 10:02:37 PM] (rkanter) HADOOP-15853. TestConfigurationDeprecation 
leaves behind a temp file,
[Oct 15, 2018 11:45:08 PM] (arp) HDDS-490. Improve om and scm start up options 
. Contributed by Namit




-1 overall


The following subsystems voted -1:
asflicense findbugs hadolint pathlen unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

XML :

   Parsing Error(s): 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/public/crossdomain.xml
 

FindBugs :

   module:hadoop-hdfs-project/hadoop-hdfs 
   Dead store to state in 
org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Saver.save(OutputStream,
 INodeSymlink) At 
FSImageFormatPBINode.java:org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Saver.save(OutputStream,
 INodeSymlink) At FSImageFormatPBINode.java:[line 663] 

Failed CTEST tests :

   test_test_libhdfs_threaded_hdfs_static 
   test_libhdfs_threaded_hdfspp_test_shim_static 

Failed junit tests :

   hadoop.hdfs.protocol.TestLayoutVersion 
   hadoop.yarn.server.nodemanager.containermanager.TestContainerManager 
   
hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage
 
   hadoop.yarn.applications.distributedshell.TestDistributedShell 
   hadoop.fs.azure.metrics.TestNativeAzureFileSystemMetricsSystem 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/928/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/928/artifact/out/diff-compile-javac-root.txt
  [300K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/928/artifact/out/diff-checkstyle-root.txt
  [17M]

   hadolint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/928/artifact/out/diff-patch-hadolint.txt
  [4.0K]

   pathlen:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/928/artifact/out/pathlen.txt
  [12K]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/928/artifact/out/diff-patch-pylint.txt
  [40K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/928/artifact/out/diff-patch-shellcheck.txt
  [20K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/928/artifact/out/diff-patch-shelldocs.txt
  [12K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/928/artifact/out/whitespace-eol.txt
  [9.3M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/928/artifact/out/whitespace-tabs.txt
  [1.1M]

   xml:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/928/artifact/out/xml.txt
  [4.0K]

   findbugs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/928/artifact/out/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/928/artifact/out/branch-findbugs-hadoop-hdds_client.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/928/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/928/artifact/out/branch-findbugs-hadoop-hdds_framework.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/928/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/928/artifact/out/branch-findbugs-hadoop-hdds_tools.txt
  [4.0K]
   
https://builds.apache.org/job/h

[jira] [Created] (YARN-8891) Documentation of the pluggable device framework

2018-10-16 Thread Zhankun Tang (JIRA)
Zhankun Tang created YARN-8891:
--

 Summary: Documentation of the pluggable device framework
 Key: YARN-8891
 URL: https://issues.apache.org/jira/browse/YARN-8891
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhankun Tang
Assignee: Zhankun Tang






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org



[jira] [Created] (YARN-8890) Port existing GPU module into pluggable device framework

2018-10-16 Thread Zhankun Tang (JIRA)
Zhankun Tang created YARN-8890:
--

 Summary: Port existing GPU module into pluggable device framework
 Key: YARN-8890
 URL: https://issues.apache.org/jira/browse/YARN-8890
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhankun Tang
Assignee: Zhankun Tang


Once we get pluggable device framework mature, we can port existing GPU related 
code into this new framework.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org



[jira] [Created] (YARN-8889) Added well-defined interface in container-executor to support vendor plugins isolation request

2018-10-16 Thread Zhankun Tang (JIRA)
Zhankun Tang created YARN-8889:
--

 Summary: Added well-defined interface in container-executor to 
support vendor plugins isolation request
 Key: YARN-8889
 URL: https://issues.apache.org/jira/browse/YARN-8889
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhankun Tang


Because of different container runtime, the isolation request from vendor 
device plugin may be raised before container launch (cgroups operations) or at 
container launch (Docker runtime).

An easy to use interface in container-executor should be provided to support 
above requirements.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org



[jira] [Created] (YARN-8888) Support device topology scheduling

2018-10-16 Thread Zhankun Tang (JIRA)
Zhankun Tang created YARN-:
--

 Summary: Support device topology scheduling
 Key: YARN-
 URL: https://issues.apache.org/jira/browse/YARN-
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhankun Tang
Assignee: Zhankun Tang


An easy way for vendor plugin to describe topology information should be 
provided in Device spec and the topology information will be used in the device 
shared local scheduler to boost performance



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org



[jira] [Created] (YARN-8887) Support isolation in pluggable device framework

2018-10-16 Thread Zhankun Tang (JIRA)
Zhankun Tang created YARN-8887:
--

 Summary: Support isolation in pluggable device framework
 Key: YARN-8887
 URL: https://issues.apache.org/jira/browse/YARN-8887
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhankun Tang
Assignee: Zhankun Tang


Devices isolation needs a complete description in API specs and a translator in 
the adapter to convert the requirements into uniform parameters passed to 
native container-executor. It should support both cgroups and Docker runtime.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org



[jira] [Created] (YARN-8886) Support RM API to query aggregated allocation across cluster

2018-10-16 Thread Zhankun Tang (JIRA)
Zhankun Tang created YARN-8886:
--

 Summary: Support RM API to query aggregated allocation across 
cluster
 Key: YARN-8886
 URL: https://issues.apache.org/jira/browse/YARN-8886
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhankun Tang
Assignee: Zhankun Tang


Query device resource allocation from NMs one by one is not efficient, RM 
supports a REST API would be better.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org



[jira] [Created] (YARN-8885) Support NM APIs to query device resource allocation

2018-10-16 Thread Zhankun Tang (JIRA)
Zhankun Tang created YARN-8885:
--

 Summary: Support NM APIs to query device resource allocation
 Key: YARN-8885
 URL: https://issues.apache.org/jira/browse/YARN-8885
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhankun Tang
Assignee: Zhankun Tang


Supprot REST API in NM for user to query allocation

*_nodemanager_address:port/ws/v1/node/resources/\{resource_name}_*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org



[jira] [Created] (YARN-8884) Support monitoring of device resource through plugin API

2018-10-16 Thread Zhankun Tang (JIRA)
Zhankun Tang created YARN-8884:
--

 Summary: Support monitoring of device resource through plugin API
 Key: YARN-8884
 URL: https://issues.apache.org/jira/browse/YARN-8884
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhankun Tang
Assignee: Zhankun Tang


In the current design, the device resource count is reported by plugin when NM 
starts but won't got update even when there're devices broken. We should 
support monitoring and update the device resource.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org



[jira] [Created] (YARN-8883) Provide an example of fake vendor plugin

2018-10-16 Thread Zhankun Tang (JIRA)
Zhankun Tang created YARN-8883:
--

 Summary: Provide an example of fake vendor plugin
 Key: YARN-8883
 URL: https://issues.apache.org/jira/browse/YARN-8883
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhankun Tang
Assignee: Zhankun Tang






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org



[jira] [Created] (YARN-8882) Add a shared device local scheduler for device plugin to use

2018-10-16 Thread Zhankun Tang (JIRA)
Zhankun Tang created YARN-8882:
--

 Summary: Add a shared device local scheduler for device plugin to 
use
 Key: YARN-8882
 URL: https://issues.apache.org/jira/browse/YARN-8882
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhankun Tang
Assignee: Zhankun Tang


Since quite a few devices uses FIFO policy to assign devices to the container, 
we use a shared schedule to handle all types of devices.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org



[jira] [Created] (YARN-8881) Add basic pluggable device plugin framework

2018-10-16 Thread Zhankun Tang (JIRA)
Zhankun Tang created YARN-8881:
--

 Summary: Add basic pluggable device plugin framework
 Key: YARN-8881
 URL: https://issues.apache.org/jira/browse/YARN-8881
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhankun Tang
Assignee: Zhankun Tang


It includes adding support in "ResourcePluginManager" to enable the framework 
and load plugin classes based on configuration, an interface for the vendor to 
implement and the adapter to decouple plugin and YARN internals.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org



[jira] [Created] (YARN-8880) Add configurations for pluggable plugin framework

2018-10-16 Thread Zhankun Tang (JIRA)
Zhankun Tang created YARN-8880:
--

 Summary: Add configurations for pluggable plugin framework
 Key: YARN-8880
 URL: https://issues.apache.org/jira/browse/YARN-8880
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Zhankun Tang
Assignee: Zhankun Tang


Added two configurations for the pluggable device framework.
{code:java}

 
yarn.nodemanager.resource-plugins.pluggable-device-framework.enable
 true/false
 
 
 yarn.nodemanager.resource-plugins.pluggable-class
 
 {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org