Re: [openstack-dev] [cyborg][glance][nova]cyborg FPGA management flow disscusion.
Hi all. This is the agent of today’s discussion. https://etherpad.openstack.org/p/cyborg-nova-poc BR Feng, Shaohe From: Zhipeng Huang [mailto:zhipengh...@gmail.com] Sent: 2018年3月8日 12:08 To: Feng, Shaohe Cc: openstack-dev@lists.openstack.org; openstack-operat...@lists.openstack.org; Du, Dolpher ; Ding, Jian-feng ; Sun, Yih Leong ; Nadathur, Sundar ; Dutch ; Rushil Chugh ; Nguyen Hung Phuong ; Justin Kilpatrick ; Ranganathan, Shobha ; zhuli ; bao.yum...@zte.com.cn; Li Liu ; xiaodong...@tencent.com; kong.w...@zte.com.cn; li.xia...@zte.com.cn Subject: Re: [openstack-dev][cyborg][glance][nova]cyborg FPGA management flow disscusion. Thanks Shaohe, Let's schedule a video conf session next week. On Thu, Mar 8, 2018 at 11:41 AM, Feng, Shaohe mailto:shaohe.f...@intel.com>> wrote: Hi All: The POC is here: https://github.com/shaohef/cyborg BR Shaohe Feng _ From: Feng, Shaohe Sent: 2018年2月12日 15:06 To: openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>; openstack-operat...@lists.openstack.org<mailto:openstack-operat...@lists.openstack.org> Cc: Du, Dolpher mailto:dolpher...@intel.com>>; Zhipeng Huang mailto:zhipengh...@gmail.com>>; Ding, Jian-feng mailto:jian-feng.d...@intel.com>>; Sun, Yih Leong mailto:yih.leong@intel.com>>; Nadathur, Sundar mailto:sundar.nadat...@intel.com>>; Dutch mailto:dalth...@xilinx.com>>; Rushil Chugh mailto:rushil.ch...@gmail.com>>; Nguyen Hung Phuong mailto:phuon...@vn.fujitsu.com>>; Justin Kilpatrick mailto:jkilp...@redhat.com>>; Ranganathan, Shobha mailto:shobha.ranganat...@intel.com>>; zhuli mailto:zhul...@huawei.com>>; bao.yum...@zte.com.cn<mailto:bao.yum...@zte.com.cn>; xiaodong...@tencent.com<mailto:xiaodong...@tencent.com>; kong.w...@zte.com.cn<mailto:kong.w...@zte.com.cn>; li.xia...@zte.com.cn<mailto:li.xia...@zte.com.cn>; Feng, Shaohe mailto:shaohe.f...@intel.com>> Subject: [openstack-dev][cyborg][glance][nova]cyborg FPGA management flow disscusion. Now I am working on an FPGA management POC with Dolpher. We have finished some code, and have discussion with Li Liu and some cyborg developer guys. Here are some discussions: image management 1. User should upload the FPGA image to glance and set the tags as follow: There are two suggestions to upload an FPGA image. A. use raw glance api like: $ openstack image create --file mypath/FPGA.img fpga.img $ openstack image set --tag FPGA --property vendor=intel --property type=crypto 58b813db-1fb7-43ec-b85c-3b771c685d22 The image must have "FPGA" tag and accelerator type(such as type=crypto). B. cyborg support a new api to upload a image. This API will wrap glance api and include the above steps, also make image record in it's local DB. 2. Cyborg agent/conductor get the FPGA image info from glance. There are also two suggestions to get the FPGA image info. A. use raw glance api. Cyborg will get the images by FPGA tag and timestamp periodically and store them in it's local cache. It will use the images tags and properties to form placement taits and resource_class name. B. store the imformations when call cybort's new upload API. 3. Image download. call glance image download API to local file. and make a corresponding md5 files for checksum. GAP in image management: missing related glance image client in cyborg. resource report management for scheduler. 1. Cyborg agent/conductor need synthesize all useful information from FPGA driver and image information. The traits will be like: CUSTOM_FPGA, CUSTOM_ACCELERATOR_CRYPTO, The resource_class will be like: CUSTOM_FPGA_INTEL_PF, CUSTOM_FPGA_INTEL_VF {"inventories": "CUSTOM_FPGA_INTEL_PF": { "allocation_ratio": 1.0, "max_unit": 4, "min_unit": 1, "reserved": 0, "step_size": 1, "total": 4 } } Accelerator claim and release: 1. Cybort will support the releated API for accelerator claim and release. It can pass the follow parameters: nodename: Which host that accelerator located on, it is required. type: This accelerator type, cyborg can get image uuid by it. it is optional. image uuid: the uuid of FPGA bitstream image, . it is optional. traits: the traits info that cyborg reports to placement. resource_class: the resource_class name that reports to placement. And return the address for the accelerator. At present, it is the PCIE_ADDRESS. 2. When claim an accelerator, type and image is None, cybort will not program the fpga for user. FPGA accelerator program API: We still need to support an independent program API for some specific scenarios. Such as as a FPGA developer, I will change my verilog logical frequently and need to do verification on my guest. I upload my new bitstrea
Re: [openstack-dev] [cyborg][glance][nova]cyborg FPGA management flow disscusion.
Thanks Shaohe, Let's schedule a video conf session next week. On Thu, Mar 8, 2018 at 11:41 AM, Feng, Shaohe wrote: > Hi All: > > The POC is here: > *https://github.com/shaohef/cyborg* <https://github.com/shaohef/cyborg> > > BR > Shaohe Feng > > _ > *From:* Feng, Shaohe > *Sent:* 2018年2月12日 15:06 > *To:* openstack-dev@lists.openstack.org; openstack-operators@lists. > openstack.org > *Cc:* Du, Dolpher ; Zhipeng Huang < > zhipengh...@gmail.com>; Ding, Jian-feng ; Sun, > Yih Leong ; Nadathur, Sundar < > sundar.nadat...@intel.com>; Dutch ; Rushil Chugh < > rushil.ch...@gmail.com>; Nguyen Hung Phuong ; > Justin Kilpatrick ; Ranganathan, Shobha < > shobha.ranganat...@intel.com>; zhuli ; > bao.yum...@zte.com.cn; xiaodong...@tencent.com; kong.w...@zte.com.cn; > li.xia...@zte.com.cn; Feng, Shaohe > *Subject:* [openstack-dev][cyborg][glance][nova]cyborg FPGA management > flow disscusion. > > > Now I am working on an FPGA management POC with Dolpher. > We have finished some code, and have discussion with Li Liu and some > cyborg developer guys. > > Here are some discussions: > > image management > 1. User should upload the FPGA image to glance and set the tags as follow: > There are two suggestions to upload an FPGA image. > A. use raw glance api like: >$ openstack image create --file mypath/FPGA.img fpga.img >$ openstack image set --tag FPGA --property vendor=intel --property > type=crypto 58b813db-1fb7-43ec-b85c-3b771c685d22 >The image must have "FPGA" tag and accelerator type(such as > type=crypto). > B. cyborg support a new api to upload a image. >This API will wrap glance api and include the above steps, also make > image record in it's local DB. > > 2. Cyborg agent/conductor get the FPGA image info from glance. > There are also two suggestions to get the FPGA image info. > A. use raw glance api. > Cyborg will get the images by FPGA tag and timestamp periodically and > store them in it's local cache. > It will use the images tags and properties to form placement taits and > resource_class name. > B. store the imformations when call cybort's new upload API. > > 3. Image download. > call glance image download API to local file. and make a corresponding md5 > files for checksum. > > GAP in image management: > missing related glance image client in cyborg. > > resource report management for scheduler. > 1. Cyborg agent/conductor need synthesize all useful information from > FPGA driver and image information. > The traits will be like: > CUSTOM_FPGA, CUSTOM_ACCELERATOR_CRYPTO, > The resource_class will be like: > CUSTOM_FPGA_INTEL_PF, CUSTOM_FPGA_INTEL_VF > {"inventories": > "CUSTOM_FPGA_INTEL_PF": { > "allocation_ratio": 1.0, > "max_unit": 4, > "min_unit": 1, > "reserved": 0, > "step_size": 1, > "total": 4 > } > } > > > Accelerator claim and release: > 1. Cybort will support the releated API for accelerator claim and release. > It can pass the follow parameters: > nodename: Which host that accelerator located on, it is required. > type: This accelerator type, cyborg can get image uuid by it. it is > optional. > image uuid: the uuid of FPGA bitstream image, . it is optional. > traits: the traits info that cyborg reports to placement. > resource_class: the resource_class name that reports to placement. > And return the address for the accelerator. At present, it is the > PCIE_ADDRESS. > 2. When claim an accelerator, type and image is None, cybort will not > program the fpga for user. > > FPGA accelerator program API: > We still need to support an independent program API for some specific > scenarios. > Such as as a FPGA developer, I will change my verilog logical frequently > and need to do verification on my guest. > I upload my new bitstream image to glance, and call cyborg to program my > FPGA accelerator. > > End user operations follow: > 1. upload an bitstream image to glance if necessary and set its tags(at > least FPGA is requied) and property. >sucn as: --tag FPGA --property vendor=intel --property type=crypto > 2. list the FPGA related traits and resource_class names by placement API. >such as get "CUSTOM_FPGA_INTEL_PF" resource_class names and > "CUSTOM_HW_INTEL,CUSTOM_HW_CRYPTO" traits. > 3. create a new falvor wiht his expected traits and resource_class as > extra spec. >such as: >"resourcesn:CUSTOM_FPGA_INTEL_PF=2" n is an integer or empty >
Re: [openstack-dev] [cyborg][glance][nova]cyborg FPGA management flow disscusion.
Hi All: The POC is here: https://github.com/shaohef/cyborg BR Shaohe Feng _ From: Feng, Shaohe Sent: 2018年2月12日 15:06 To: openstack-dev@lists.openstack.org; openstack-operat...@lists.openstack.org Cc: Du, Dolpher ; Zhipeng Huang ; Ding, Jian-feng ; Sun, Yih Leong ; Nadathur, Sundar ; Dutch ; Rushil Chugh ; Nguyen Hung Phuong ; Justin Kilpatrick ; Ranganathan, Shobha ; zhuli ; bao.yum...@zte.com.cn; xiaodong...@tencent.com; kong.w...@zte.com.cn; li.xia...@zte.com.cn; Feng, Shaohe Subject: [openstack-dev][cyborg][glance][nova]cyborg FPGA management flow disscusion. Now I am working on an FPGA management POC with Dolpher. We have finished some code, and have discussion with Li Liu and some cyborg developer guys. Here are some discussions: image management 1. User should upload the FPGA image to glance and set the tags as follow: There are two suggestions to upload an FPGA image. A. use raw glance api like: $ openstack image create --file mypath/FPGA.img fpga.img $ openstack image set --tag FPGA --property vendor=intel --property type=crypto 58b813db-1fb7-43ec-b85c-3b771c685d22 The image must have "FPGA" tag and accelerator type(such as type=crypto). B. cyborg support a new api to upload a image. This API will wrap glance api and include the above steps, also make image record in it's local DB. 2. Cyborg agent/conductor get the FPGA image info from glance. There are also two suggestions to get the FPGA image info. A. use raw glance api. Cyborg will get the images by FPGA tag and timestamp periodically and store them in it's local cache. It will use the images tags and properties to form placement taits and resource_class name. B. store the imformations when call cybort's new upload API. 3. Image download. call glance image download API to local file. and make a corresponding md5 files for checksum. GAP in image management: missing related glance image client in cyborg. resource report management for scheduler. 1. Cyborg agent/conductor need synthesize all useful information from FPGA driver and image information. The traits will be like: CUSTOM_FPGA, CUSTOM_ACCELERATOR_CRYPTO, The resource_class will be like: CUSTOM_FPGA_INTEL_PF, CUSTOM_FPGA_INTEL_VF {"inventories": "CUSTOM_FPGA_INTEL_PF": { "allocation_ratio": 1.0, "max_unit": 4, "min_unit": 1, "reserved": 0, "step_size": 1, "total": 4 } } Accelerator claim and release: 1. Cybort will support the releated API for accelerator claim and release. It can pass the follow parameters: nodename: Which host that accelerator located on, it is required. type: This accelerator type, cyborg can get image uuid by it. it is optional. image uuid: the uuid of FPGA bitstream image, . it is optional. traits: the traits info that cyborg reports to placement. resource_class: the resource_class name that reports to placement. And return the address for the accelerator. At present, it is the PCIE_ADDRESS. 2. When claim an accelerator, type and image is None, cybort will not program the fpga for user. FPGA accelerator program API: We still need to support an independent program API for some specific scenarios. Such as as a FPGA developer, I will change my verilog logical frequently and need to do verification on my guest. I upload my new bitstream image to glance, and call cyborg to program my FPGA accelerator. End user operations follow: 1. upload an bitstream image to glance if necessary and set its tags(at least FPGA is requied) and property. sucn as: --tag FPGA --property vendor=intel --property type=crypto 2. list the FPGA related traits and resource_class names by placement API. such as get "CUSTOM_FPGA_INTEL_PF" resource_class names and "CUSTOM_HW_INTEL,CUSTOM_HW_CRYPTO" traits. 3. create a new falvor wiht his expected traits and resource_class as extra spec. such as: "resourcesn:CUSTOM_FPGA_INTEL_PF=2" n is an integer or empty string. "required:CUSTOM_HW_INTEL,CUSTOM_HW_CRYPTO". 4. create the VM with this flavor. BR Shaohe Feng __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [cyborg][glance][nova]cyborg FPGA management flow disscusion.
Now I am working on an FPGA management POC with Dolpher. We have finished some code, and have discussion with Li Liu and some cyborg developer guys. Here are some discussions: image management 1. User should upload the FPGA image to glance and set the tags as follow: There are two suggestions to upload an FPGA image. A. use raw glance api like: $ openstack image create --file mypath/FPGA.img fpga.img $ openstack image set --tag FPGA --property vendor=intel --property type=crypto 58b813db-1fb7-43ec-b85c-3b771c685d22 The image must have "FPGA" tag and accelerator type(such as type=crypto). B. cyborg support a new api to upload a image. This API will wrap glance api and include the above steps, also make image record in it's local DB. 2. Cyborg agent/conductor get the FPGA image info from glance. There are also two suggestions to get the FPGA image info. A. use raw glance api. Cyborg will get the images by FPGA tag and timestamp periodically and store them in it's local cache. It will use the images tags and properties to form placement taits and resource_class name. B. store the imformations when call cybort's new upload API. 3. Image download. call glance image download API to local file. and make a corresponding md5 files for checksum. GAP in image management: missing related glance image client in cyborg. resource report management for scheduler. 1. Cyborg agent/conductor need synthesize all useful information from FPGA driver and image information. The traits will be like: CUSTOM_FPGA, CUSTOM_ACCELERATOR_CRYPTO, The resource_class will be like: CUSTOM_FPGA_INTEL_PF, CUSTOM_FPGA_INTEL_VF {"inventories": "CUSTOM_FPGA_INTEL_PF": { "allocation_ratio": 1.0, "max_unit": 4, "min_unit": 1, "reserved": 0, "step_size": 1, "total": 4 } } Accelerator claim and release: 1. Cybort will support the releated API for accelerator claim and release. It can pass the follow parameters: nodename: Which host that accelerator located on, it is required. type: This accelerator type, cyborg can get image uuid by it. it is optional. image uuid: the uuid of FPGA bitstream image, . it is optional. traits: the traits info that cyborg reports to placement. resource_class: the resource_class name that reports to placement. And return the address for the accelerator. At present, it is the PCIE_ADDRESS. 2. When claim an accelerator, type and image is None, cybort will not program the fpga for user. FPGA accelerator program API: We still need to support an independent program API for some specific scenarios. Such as as a FPGA developer, I will change my verilog logical frequently and need to do verification on my guest. I upload my new bitstream image to glance, and call cyborg to program my FPGA accelerator. End user operations follow: 1. upload an bitstream image to glance if necessary and set its tags(at least FPGA is requied) and property. sucn as: --tag FPGA --property vendor=intel --property type=crypto 2. list the FPGA related traits and resource_class names by placement API. such as get "CUSTOM_FPGA_INTEL_PF" resource_class names and "CUSTOM_HW_INTEL,CUSTOM_HW_CRYPTO" traits. 3. create a new falvor wiht his expected traits and resource_class as extra spec. such as: "resourcesn:CUSTOM_FPGA_INTEL_PF=2" n is an integer or empty string. "required:CUSTOM_HW_INTEL,CUSTOM_HW_CRYPTO". 4. create the VM with this flavor. BR Shaohe Feng __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev