[jira] [Commented] (YARN-8563) [Submarine] Support users to specify Python/TF package/version/dependencies for training job.

2018-08-24 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592475#comment-16592475 ] Zhankun Tang commented on YARN-8563: [~leftnoteasy] One question: *When and how will the prebuilt

[jira] [Commented] (YARN-8698) [Submarine] Failed to add hadoop dependencies in docker container when submitting a submarine job

2018-08-24 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591265#comment-16591265 ] Zhankun Tang commented on YARN-8698: [~yuan_zac] Yeah. I guess so. I haven't reproduce your issue with

[jira] [Commented] (YARN-8698) [Submarine] Failed to add hadoop dependencies in docker container when submitting a submarine job

2018-08-23 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591120#comment-16591120 ] Zhankun Tang commented on YARN-8698: [~yuan_zac] Yeah. Thanks for clarification. Actually I’ve tried

[jira] [Commented] (YARN-8698) [Submarine] Failed to add hadoop dependencies in docker container when submitting a submarine job

2018-08-22 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16589682#comment-16589682 ] Zhankun Tang commented on YARN-8698: [~yuan_zac] Yeah. Wrong HADOOP_COMMON_HOME env will cause "hadoop

[jira] [Commented] (YARN-8561) [Submarine] Initial implementation: Training job submission and job history retrieval

2018-08-22 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16589561#comment-16589561 ] Zhankun Tang commented on YARN-8561: [~leftnoteasy] I'm going through the code. And a minor problem

[jira] [Commented] (YARN-8698) [Submarine] Failed to add hadoop dependencies in docker container when submitting a submarine job

2018-08-22 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16589498#comment-16589498 ] Zhankun Tang commented on YARN-8698: [~yuan_zac] Thanks for the path! And I have a question that does

[jira] [Updated] (YARN-8456) Fix a configuration handling bug when user leave FPGA discover executable path configuration default but set OpenCL SDK path environment variable

2018-06-23 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-8456: --- Summary: Fix a configuration handling bug when user leave FPGA discover executable path configuration

[jira] [Updated] (YARN-8456) Fix a bug when user leave FPGA discover executable path configuration default but set OpenCL SDK path environment variable

2018-06-23 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-8456: --- Description: *Issue:* When the user doesn't configure

[jira] [Comment Edited] (YARN-8456) Fix a bug when user leave FPGA discover executable path configuration default but set OpenCL SDK path environment variable

2018-06-23 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521026#comment-16521026 ] Zhankun Tang edited comment on YARN-8456 at 6/23/18 9:56 AM: - It seems ok in

[jira] [Updated] (YARN-8456) Fix a bug when user leave FPGA discover executable path configuration default but set OpenCL SDK path environment variable

2018-06-23 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-8456: --- Attachment: (was: YARN-8456-trunk.001.path) > Fix a bug when user leave FPGA discover executable

[jira] [Updated] (YARN-8456) Fix a bug when user leave FPGA discover executable path configuration default but set OpenCL SDK path environment variable

2018-06-23 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-8456: --- Attachment: YARN-8456-trunk.001.patch > Fix a bug when user leave FPGA discover executable path

[jira] [Commented] (YARN-8456) Fix a bug when user leave FPGA discover executable path configuration default but set OpenCL SDK path environment variable

2018-06-23 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521026#comment-16521026 ] Zhankun Tang commented on YARN-8456: It seems ok in the unit tests and I also tested in my development

[jira] [Updated] (YARN-8456) Fix a bug when user leave FPGA discover executable path configuration default but set OpenCL SDK path environment variable

2018-06-23 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-8456: --- Description: *Issue:* When the user doesn't configure

[jira] [Updated] (YARN-8456) Fix a bug when user leave FPGA discover executable path configuration default but set OpenCL SDK path environment variable

2018-06-23 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-8456: --- Attachment: YARN-8456-trunk.001.path > Fix a bug when user leave FPGA discover executable path

[jira] [Updated] (YARN-8456) Fix a bug when user leave FPGA discover executable path configuration default but set OpenCL SDK path environment variable

2018-06-22 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-8456: --- Summary: Fix a bug when user leave FPGA discover executable path configuration default but set OpenCL

[jira] [Created] (YARN-8456) Fix a bug when user leave FPGA discover executable path configuration empty but set OpenCL SDK path environment variable

2018-06-22 Thread Zhankun Tang (JIRA)
Zhankun Tang created YARN-8456: -- Summary: Fix a bug when user leave FPGA discover executable path configuration empty but set OpenCL SDK path environment variable Key: YARN-8456 URL:

[jira] [Assigned] (YARN-8456) Fix a bug when user leave FPGA discover executable path configuration empty but set OpenCL SDK path environment variable

2018-06-22 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang reassigned YARN-8456: -- Assignee: Zhankun Tang > Fix a bug when user leave FPGA discover executable path configuration

[jira] [Commented] (YARN-7579) Add support for FPGA information shown in webUI

2018-06-19 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516736#comment-16516736 ] Zhankun Tang commented on YARN-7579: I'd like to move this forward. But not quite sure that, which

[jira] [Updated] (YARN-7893) Document the FPGA isolation feature

2018-02-25 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-7893: --- Attachment: YARN-7893-trunk-004.patch > Document the FPGA isolation feature >

[jira] [Updated] (YARN-7893) Document the FPGA isolation feature

2018-02-25 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-7893: --- Attachment: YARN-7893-trunk-003.patch > Document the FPGA isolation feature >

[jira] [Updated] (YARN-7893) Document the FPGA isolation feature

2018-02-25 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-7893: --- Attachment: FPGA-doc-YARN-7893-v3.pdf > Document the FPGA isolation feature >

[jira] [Commented] (YARN-7893) Document the FPGA isolation feature

2018-02-25 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376399#comment-16376399 ] Zhankun Tang commented on YARN-7893: [~leftnoteasy] , one quick question, we all know that when a user 

[jira] [Commented] (YARN-7893) Document the FPGA isolation feature

2018-02-25 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376332#comment-16376332 ] Zhankun Tang commented on YARN-7893: [~leftnoteasy] , really sorry that I missed this comment during

[jira] [Updated] (YARN-7893) Document the FPGA isolation feature

2018-02-07 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-7893: --- Attachment: YARN-7893-trunk-002.patch > Document the FPGA isolation feature >

[jira] [Assigned] (YARN-7893) Document the FPGA isolation feature

2018-02-07 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang reassigned YARN-7893: -- Assignee: Zhankun Tang > Document the FPGA isolation feature >

[jira] [Commented] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2018-02-05 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352556#comment-16352556 ] Zhankun Tang commented on YARN-6507: [~tangzhankun] , A minor issue found in IntelOpenclFPGA when YARN

[jira] [Commented] (YARN-7893) Document the FPGA isolation feature

2018-02-05 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352548#comment-16352548 ] Zhankun Tang commented on YARN-7893: [~wangda] , [~zyluo] The draft doc is attached. Please review. >

[jira] [Updated] (YARN-7893) Document the FPGA isolation feature

2018-02-05 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-7893: --- Attachment: YARN-7893-trunk-001.patch > Document the FPGA isolation feature >

[jira] [Updated] (YARN-7893) Document the FPGA isolation feature

2018-02-05 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-7893: --- Attachment: FPGA-doc-YARN-7893.pdf > Document the FPGA isolation feature >

[jira] [Created] (YARN-7893) Document the FPGA isolation feature

2018-02-05 Thread Zhankun Tang (JIRA)
Zhankun Tang created YARN-7893: -- Summary: Document the FPGA isolation feature Key: YARN-7893 URL: https://issues.apache.org/jira/browse/YARN-7893 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Updated] (YARN-7443) Add native FPGA module support to do isolation with cgroups

2017-12-05 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-7443: --- Attachment: YARN-7443-trunk.004.patch > Add native FPGA module support to do isolation with cgroups >

[jira] [Commented] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-12-03 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276359#comment-16276359 ] Zhankun Tang commented on YARN-6507: [~wangda], Thank you too. It's a good starting. :) > Add support

[jira] [Updated] (YARN-7443) Add native FPGA module support to do isolation with cgroups

2017-12-03 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-7443: --- Attachment: YARN-7443-trunk.003.patch [~wangda], sure. Added comment in c-e.cfg and updated > Add

[jira] [Updated] (YARN-5983) [Umbrella] Support for FPGA as a Resource in YARN

2017-11-28 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-5983: --- Description: As various big data workload running on YARN, CPU will no longer scale eventually and

[jira] [Created] (YARN-7579) Add support for FPGA information shown in webUI

2017-11-28 Thread Zhankun Tang (JIRA)
Zhankun Tang created YARN-7579: -- Summary: Add support for FPGA information shown in webUI Key: YARN-7579 URL: https://issues.apache.org/jira/browse/YARN-7579 Project: Hadoop YARN Issue Type:

[jira] [Updated] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-11-28 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Attachment: YARN-6507-trunk.012.patch Rebased on the trunk. Leave the new added interface

[jira] [Updated] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-11-28 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Attachment: YARN-6507-trunk.011.patch Strange that the YARN-6507-trunk.010.patch has no QA result

[jira] [Commented] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-11-28 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269937#comment-16269937 ] Zhankun Tang commented on YARN-6507: [~wangda], thanks for the review! > Add support in NodeManager to

[jira] [Updated] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-11-26 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Attachment: YARN-6507-trunk.010.patch > Add support in NodeManager to isolate FPGA devices with

[jira] [Updated] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-11-26 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Attachment: (was: YARN-6507-trunk.0010.patch) > Add support in NodeManager to isolate FPGA devices

[jira] [Updated] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-11-26 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Attachment: YARN-6507-trunk.0010.patch Rebased on trunk (2bde3aedf139368fc71f053d8dd6580b498ff46d)

[jira] [Commented] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-11-25 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16265938#comment-16265938 ] Zhankun Tang commented on YARN-6507: [~wangda], the end-to-end test reported is attached in YARN-5983.

[jira] [Updated] (YARN-5983) [Umbrella] Support for FPGA as a Resource in YARN

2017-11-25 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-5983: --- Attachment: YARN-5983_end-to-end_test_report.pdf Add an end-to-end test report for your reference. >

[jira] [Updated] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-11-21 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Attachment: YARN-6507-trunk.009.patch Fix an bug when parsing the toolchain output > Add support in

[jira] [Commented] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-11-21 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261930#comment-16261930 ] Zhankun Tang commented on YARN-6507: [~wangda], Thanks for the reply. Yeah. Of course. I'll summarize

[jira] [Updated] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-11-21 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Attachment: YARN-6507-trunk.008.patch > Add support in NodeManager to isolate FPGA devices with

[jira] [Updated] (YARN-5983) [Umbrella] Support for FPGA as a Resource in YARN

2017-11-14 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-5983: --- Attachment: YARN-5983-implementation-notes.pdf Add a design and implementation note for YARN-6507 and

[jira] [Updated] (YARN-7443) Add native FPGA module support to do isolation with cgroups

2017-11-13 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-7443: --- Attachment: YARN-7443-trunk.002.patch fix same type define function in FPGA module when building in

[jira] [Updated] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-11-13 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Attachment: YARN-6507-trunk.007.patch fix the white space > Add support in NodeManager to isolate

[jira] [Updated] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-11-13 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Attachment: YARN-6507-trunk.006.patch Add configuration and abstraction for vendor FPGA plugin > Add

[jira] [Updated] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-11-11 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Attachment: YARN-6507-trunk.005.patch The 005 patch mainly includes bug-fix of zero FPGA devices and

[jira] [Updated] (YARN-7443) Add native FPGA module support to do isolation with cgroups

2017-11-05 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-7443: --- Attachment: YARN-7443-trunk.001.patch Draft patch > Add native FPGA module support to do isolation

[jira] [Created] (YARN-7443) Add native FPGA module support to do isolation with cgroups

2017-11-05 Thread Zhankun Tang (JIRA)
Zhankun Tang created YARN-7443: -- Summary: Add native FPGA module support to do isolation with cgroups Key: YARN-7443 URL: https://issues.apache.org/jira/browse/YARN-7443 Project: Hadoop YARN

[jira] [Commented] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-11-01 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234059#comment-16234059 ] Zhankun Tang commented on YARN-6507: [~wangda], WIP FPGA resource plugin is finished. The native FPGA

[jira] [Updated] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-11-01 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Attachment: YARN-6507-trunk.004.patch Seems unrelated unit test failure. Try again. > Add support in

[jira] [Updated] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-10-31 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Attachment: YARN-6507-trunk.003.patch > Add support in NodeManager to isolate FPGA devices with

[jira] [Updated] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-10-31 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Attachment: YARN-6507-trunk.002.patch Fix some findbug warnings > Add support in NodeManager to

[jira] [Updated] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-10-30 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Fix Version/s: (was: YARN-3926) > Add support in NodeManager to isolate FPGA devices with CGroups

[jira] [Updated] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-10-30 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Attachment: YARN-6507-trunk.001.patch Draft patch for FPGA java side code > Add support in

[jira] [Commented] (YARN-6508) Support FPGA plugin

2017-10-30 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226162#comment-16226162 ] Zhankun Tang commented on YARN-6508: The default vendor specific plugin is implemented in YARN-6507. So

[jira] [Resolved] (YARN-6508) Support FPGA plugin

2017-10-30 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang resolved YARN-6508. Resolution: Implemented > Support FPGA plugin > --- > > Key:

[jira] [Updated] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-10-30 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Description: Support local FPGA resource scheduler to assign/isolate N FPGA slots to a container. At

[jira] [Commented] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-10-30 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226156#comment-16226156 ] Zhankun Tang commented on YARN-6507: Because we'll first support one kind of plugin for the time being.

[jira] [Updated] (YARN-6507) Add support in NodeManager to isolate FPGA devices with CGroups

2017-10-30 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Summary: Add support in NodeManager to isolate FPGA devices with CGroups (was: Support FPGA

[jira] [Commented] (YARN-6620) Add support in NodeManager to isolate GPU devices by using CGroups

2017-10-18 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208861#comment-16208861 ] Zhankun Tang commented on YARN-6620: [~wangda], thanks for the clarification. The below code confuses

[jira] [Comment Edited] (YARN-6620) Add support in NodeManager to isolate GPU devices by using CGroups

2017-10-16 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207026#comment-16207026 ] Zhankun Tang edited comment on YARN-6620 at 10/17/17 5:43 AM: -- [~wangda],

[jira] [Commented] (YARN-6620) Add support in NodeManager to isolate GPU devices by using CGroups

2017-10-16 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207026#comment-16207026 ] Zhankun Tang commented on YARN-6620: [~wangda], Thanks for the great effort. I'll implement the FPGA

[jira] [Commented] (YARN-6620) [YARN-6223] NM Java side code changes to support isolate GPU devices by using CGroups

2017-09-18 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16171026#comment-16171026 ] Zhankun Tang commented on YARN-6620: [~wangda], Thanks for the 008 patch. LGTM. > [YARN-6223] NM Java

[jira] [Commented] (YARN-6620) [YARN-6223] NM Java side code changes to support isolate GPU devices by using CGroups

2017-09-18 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169655#comment-16169655 ] Zhankun Tang commented on YARN-6620: {quote} Good point, I think we should use node attribute to

[jira] [Commented] (YARN-6620) [YARN-6223] NM Java side code changes to support isolate GPU devices by using CGroups

2017-09-17 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169566#comment-16169566 ] Zhankun Tang commented on YARN-6620: [~wangda], Thanks for the patch! Now we have defined the resource

[jira] [Commented] (YARN-6852) [YARN-6223] Native code changes to support isolate GPU devices by using CGroups

2017-09-12 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164150#comment-16164150 ] Zhankun Tang commented on YARN-6852: [~wangda], Yeah. At present, all supported resource type needs to

[jira] [Commented] (YARN-6852) [YARN-6223] Native code changes to support isolate GPU devices by using CGroups

2017-09-11 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16162318#comment-16162318 ] Zhankun Tang commented on YARN-6852: [~wangda], agree with you. I'll refactor both native and java side

[jira] [Commented] (YARN-6852) [YARN-6223] Native code changes to support isolate GPU devices by using CGroups

2017-09-08 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158338#comment-16158338 ] Zhankun Tang commented on YARN-6852: [~wangda], Thanks for the patch. I'd like to update FPGA related

[jira] [Commented] (YARN-6852) [YARN-6223] Native code changes to support isolate GPU devices by using CGroups

2017-08-07 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117773#comment-16117773 ] Zhankun Tang commented on YARN-6852: [~miklos.szeg...@cloudera.com], [~wangda], Thanks for the good

[jira] [Commented] (YARN-6223) [Umbrella] Natively support GPU configuration/discovery/scheduling/isolation on YARN

2017-07-19 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093148#comment-16093148 ] Zhankun Tang commented on YARN-6223: [~wangda], sorry for the late reply. Great thanks for the ver.3

[jira] [Comment Edited] (YARN-6223) [Umbrella] Natively support GPU configuration/discovery/scheduling/isolation on YARN

2017-07-19 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093148#comment-16093148 ] Zhankun Tang edited comment on YARN-6223 at 7/19/17 2:19 PM: - [~wangda], sorry

[jira] [Commented] (YARN-6507) Support FPGA abstraction framework on NM side

2017-07-19 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093079#comment-16093079 ] Zhankun Tang commented on YARN-6507: [~wangda], Yeah. agree with that the recovery module in NM is a

[jira] [Updated] (YARN-6507) Support FPGA abstraction framework on NM side

2017-07-19 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Attachment: YARN-6507-branch-YARN-3926.002.patch Decouple the vendor FPGA plugin from the framework.

[jira] [Commented] (YARN-6720) Support updating FPGA related constraint node label after FPGA device re-configuration

2017-07-13 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086747#comment-16086747 ] Zhankun Tang commented on YARN-6720: [~Naganarasimha], sorry for the late reply. Yeah. So far, I can

[jira] [Commented] (YARN-6720) Support updating FPGA related constraint node label after FPGA device re-configuration

2017-07-06 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077524#comment-16077524 ] Zhankun Tang commented on YARN-6720: [~wangda], I think this is depend on YARN-3409's constraint label

[jira] [Commented] (YARN-6720) Support updating FPGA related constraint node label after FPGA device re-configuration

2017-06-30 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069618#comment-16069618 ] Zhankun Tang commented on YARN-6720: [~wangda]. Maybe it's my fault. Although the reconfigure FPGA

[jira] [Assigned] (YARN-6507) Support FPGA abstraction framework on NM side

2017-06-27 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang reassigned YARN-6507: -- Assignee: Zhankun Tang > Support FPGA abstraction framework on NM side >

[jira] [Commented] (YARN-6507) Support FPGA abstraction framework on NM side

2017-06-27 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16064373#comment-16064373 ] Zhankun Tang commented on YARN-6507: Some thoughts about next step: 1. For the vendor specific plugin,

[jira] [Updated] (YARN-6507) Support FPGA abstraction framework on NM side

2017-06-26 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Fix Version/s: YARN-3926 > Support FPGA abstraction framework on NM side >

[jira] [Updated] (YARN-6507) Support FPGA abstraction framework on NM side

2017-06-26 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6507: --- Attachment: YARN-6507-branch-YARN-3926.001.patch A draft patch. Please review [~wangda] > Support

[jira] [Updated] (YARN-6720) Support updating FPGA related constraint node label after FPGA device re-configuration

2017-06-19 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6720: --- Attachment: Storing-and-Updating-extra-FPGA-resource-attributes-in-hdfs_v1.pdf The draft proposal is

[jira] [Updated] (YARN-6720) Support updating FPGA related constraint node label after FPGA device re-configuration

2017-06-18 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6720: --- External issue ID: (was: YARN-3409) > Support updating FPGA related constraint node label after FPGA

[jira] [Updated] (YARN-6720) Support updating FPGA related constraint node label after FPGA device re-configuration

2017-06-18 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-6720: --- External issue ID: YARN-3409 > Support updating FPGA related constraint node label after FPGA device

[jira] [Created] (YARN-6720) Support updating FPGA related constraint node label after FPGA device re-configuration

2017-06-18 Thread Zhankun Tang (JIRA)
Zhankun Tang created YARN-6720: -- Summary: Support updating FPGA related constraint node label after FPGA device re-configuration Key: YARN-6720 URL: https://issues.apache.org/jira/browse/YARN-6720

[jira] [Commented] (YARN-5983) [Umbrella] Support for FPGA as a Resource in YARN

2017-04-28 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988445#comment-15988445 ] Zhankun Tang commented on YARN-5983: [~devaraj.k], Thanks a lot for the review. 1. {code:xml} The

[jira] [Commented] (YARN-5983) [Umbrella] Support for FPGA as a Resource in YARN

2017-04-27 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988143#comment-15988143 ] Zhankun Tang commented on YARN-5983: [~wangda], Thanks for the review. Yes, quite agree that YARN-3409

[jira] [Created] (YARN-6508) Support Intel FPGA plugin

2017-04-20 Thread Zhankun Tang (JIRA)
Zhankun Tang created YARN-6508: -- Summary: Support Intel FPGA plugin Key: YARN-6508 URL: https://issues.apache.org/jira/browse/YARN-6508 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Created] (YARN-6507) Support FPGA abstraction framework on NM side

2017-04-20 Thread Zhankun Tang (JIRA)
Zhankun Tang created YARN-6507: -- Summary: Support FPGA abstraction framework on NM side Key: YARN-6507 URL: https://issues.apache.org/jira/browse/YARN-6507 Project: Hadoop YARN Issue Type:

[jira] [Updated] (YARN-5983) [Umbrella] Support for FPGA as a Resource in YARN

2017-04-20 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-5983: --- Attachment: YARN-5983-Support-FPGA-resource-on-NM-side_v1.pdf Uploaded the initial draft proposal of

[jira] [Comment Edited] (YARN-6223) [Umbrella] Natively support GPU configuration/discovery/scheduling/isolation on YARN

2017-04-01 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15952132#comment-15952132 ] Zhankun Tang edited comment on YARN-6223 at 4/1/17 8:54 AM: [~wangda], Yeah, I

[jira] [Commented] (YARN-6223) [Umbrella] Natively support GPU configuration/discovery/scheduling/isolation on YARN

2017-04-01 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15952132#comment-15952132 ] Zhankun Tang commented on YARN-6223: [~wangda], Yeah, I agree. It's a very good plan. > [Umbrella]

[jira] [Commented] (YARN-6043) [HDL] Tensorflow on YARN

2017-04-01 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15952113#comment-15952113 ] Zhankun Tang commented on YARN-6043: Given our TensorFlow on YARN prototype is basically working .

[jira] [Commented] (YARN-3926) Extend the YARN resource model for easier resource-type management and profiles

2017-03-31 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15952056#comment-15952056 ] Zhankun Tang commented on YARN-3926: {quote} As mentioned above, the overrides will only be allowed for

[jira] [Comment Edited] (YARN-6223) [Umbrella] Natively support GPU configuration/discovery/scheduling/isolation on YARN

2017-03-31 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950199#comment-15950199 ] Zhankun Tang edited comment on YARN-6223 at 4/1/17 3:23 AM: [~wangda], thanks

[jira] [Comment Edited] (YARN-6223) [Umbrella] Natively support GPU configuration/discovery/scheduling/isolation on YARN

2017-03-31 Thread Zhankun Tang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950199#comment-15950199 ] Zhankun Tang edited comment on YARN-6223 at 3/31/17 8:39 AM: - [~wangda], thanks

<    4   5   6   7   8   9   10   11   >