On 03/06/2018 09:36 PM, Alex Xu wrote:
2018-03-07 10:21 GMT+08:00 Alex Xu <sou...@gmail.com <mailto:sou...@gmail.com>>:




    2018-03-06 22:45 GMT+08:00 Mooney, Sean K <sean.k.moo...@intel.com
    <mailto:sean.k.moo...@intel.com>>:

        __ __

        __ __

        *From:*Matthew Booth [mailto:mbo...@redhat.com
        <mailto:mbo...@redhat.com>]
        *Sent:* Saturday, March 3, 2018 4:15 PM
        *To:* OpenStack Development Mailing List (not for usage
        questions) <openstack-dev@lists.openstack.org
        <mailto:openstack-dev@lists.openstack.org>>
        *Subject:* Re: [openstack-dev] [Nova] [Cyborg] Tracking multiple
        functions____

        __ __

        On 2 March 2018 at 14:31, Jay Pipes <jaypi...@gmail.com
        <mailto:jaypi...@gmail.com>> wrote:____

            On 03/02/2018 02:00 PM, Nadathur, Sundar wrote:____

                Hello Nova team,

                      During the Cyborg discussion at Rocky PTG, we
                proposed a flow for FPGAs wherein the request spec asks
                for a device type as a resource class, and optionally a
                function (such as encryption) in the extra specs. This
                does not seem to work well for the usage model that I’ll
                describe below.

                An FPGA device may implement more than one function. For
                example, it may implement both compression and
                encryption. Say a cluster has 10 devices of device type
                X, and each of them is programmed to offer 2 instances
                of function A and 4 instances of function B. More
                specifically, the device may implement 6 PCI functions,
                with 2 of them tied to function A, and the other 4 tied
                to function B. So, we could have 6 separate instances
                accessing functions on the same device.____

        __ __

        Does this imply that Cyborg can't reprogram the FPGA at all?____

        */[Mooney, Sean K] cyborg is intended to support fixed function
        acclerators also so it will not always be able to program the
        accelerator. In this case where an fpga is preprogramed with a
        multi function bitstream that is statically provisioned cyborge
        will not be able to reprogram the slot if any of the fuctions
        from that slot are already allocated to an instance. In this
        case it will have to treat it like a fixed function device and
        simply allocate a unused  vf  of the corret type if available.
        ____/*


        ____


                In the current flow, the device type X is modeled as a
                resource class, so Placement will count how many of them
                are in use. A flavor for ‘RC device-type-X + function A’
                will consume one instance of the RC device-type-X.  But
                this is not right because this precludes other functions
                on the same device instance from getting used.

                One way to solve this is to declare functions A and B as
                resource classes themselves and have the flavor request
                the function RC. Placement will then correctly count the
                function instances. However, there is still a problem:
                if the requested function A is not available, Placement
                will return an empty list of RPs, but we need some way
                to reprogram some device to create an instance of
                function A.____


            Clearly, nova is not going to be reprogramming devices with
            an instance of a particular function.

            Cyborg might need to have a separate agent that listens to
            the nova notifications queue and upon seeing an event that
            indicates a failed build due to lack of resources, then
            Cyborg can try and reprogram a device and then try
            rebuilding the original request.____

        __ __

        It was my understanding from that discussion that we intend to
        insert Cyborg into the spawn workflow for device configuration
        in the same way that we currently insert resources provided by
        Cinder and Neutron. So while Nova won't be reprogramming a
        device, it will be calling out to Cyborg to reprogram a device,
        and waiting while that happens.____

        My understanding is (and I concede some areas are a little
        hazy):____

        * The flavors says device type X with function Y____

        * Placement tells us everywhere with device type X____

        * A weigher orders these by devices which already have an
        available function Y (where is this metadata stored?)____

        * Nova schedules to host Z____

        * Nova host Z asks cyborg for a local function Y and blocks____

           * Cyborg hopefully returns function Y which is already
        available____

           * If not, Cyborg reprograms a function Y, then returns it____

        Can anybody correct me/fill in the gaps?____

        */[Mooney, Sean K] that correlates closely to my recollection
        also. As for the metadata I think the weigher may need to call
        to cyborg to retrieve this as it will not be available in the
        host state object./*

    Is it the nova scheduler weigher or we want to support weigh on
    placement? Function is traits as I think, so can we have
    preferred_traits? I remember we talk about that parameter in the
    past, but we don't have good use-case at that time. This is good
    use-case.


If we call the Cyborg from the nova scheduler weigher, that will slow down the scheduling a lot also.

Right, which is why I don't want to do any weighing in Placement at all. If folks want to sort by things that require long-running code/callbacks or silly temporal things like metrics, they can do that in a custom weigher in the nova-scheduler and take the performance hit there.

Best,
-jay

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to