Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[John Griffith]

On Thu, Feb 23, 2017 at 7:18 PM, Zhenyu Zheng 
wrote:

> Matt,
>
> Thanks for the information, I will check that; But still I think the user
> demand here is to use local disk from
> compute node as block device, as the data can be remained if the old vm
> got deleted, and we can start a
> new one with the data and having the performance they wanted.
>
> Kevin Zheng
>
> On Fri, Feb 24, 2017 at 4:06 AM, Matt Riedemann <
> mrie...@linux.vnet.ibm.com> wrote:
>
>> On 9/26/2016 9:21 PM, Zhenyu Zheng wrote:
>>
>>> Hi,
>>>
>>> Thanks for the reply, actually approach one is not we are looking for,
>>> our demands is to attach the real physical volume from compute node to
>>> VMs,
>>> by this way we can achieve the performance we need for usecases such as
>>> big data, this can be done by cinder using BlockDeviceDriver, it is quite
>>> different from the approach one you mentioned. The only problem now is
>>> that we cannot practially ensure the compute resource located on the same
>>> host with the volume, as Matt mentioned above, currently we have to
>>> arrange 1:1 AZ in Cinder and Nova to do this and it is not practical in
>>> commercial
>>> deployments.
>>>
>>> Thanks.
>>>
>>>
>> Kevin,
>>
>> Is the issue because you can't use ephemeral local disks (it must be a
>> persistent boot from volume)?
>>
>> Have you looked at using the LVM image backend for local storage in Nova?
>> I thought cfriesen said once that windriver is doing high performance
>> config using local LVM in nova.
>>
>> --
>>
>> Thanks,
>>
>> Matt Riedemann
>>
>>
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> Hi Kevin,

Few things that may be related to your request:

>>"our demands is to attach the real physical volume from compute node to
VMs"

Cinder had this capability for a while in the form of the block-driver, but
it's been removed due to lack of functionality, testing and really interest
all the way around.  We also took a look at performance data and the fact
was that the performance between iSCSI over a 10Gig dedicated network and a
local block device was minimal.  The block-driver model breaks just about
every Cinder feature at this point so rather than carry it around as a
special one off case it's been removed and if you really need local disk to
the compute node, you need to just use the ephemeral LVM driver in Nova.

>>"compute node as block device, as the data can be remained if the old vm
got deleted"

Yes, I understand what you are asking for here, and that's similar to how
the old block-driver worked; like I said though that's been deprecated and
removed.  I would be curious to get more info about the *requirement* here
in terms of performance.  We had a number of people look at the performance
characteristics between a Cinder LVM Volume behind an LIO Tgt and found
there to be minimal differences in performance between the raw disk device
without iSCSI.

There are ways to bring the block-driver up to feature parity etc, but
frankly most have decided it's not worth it as there's very little real
benefit to using it over the existing and well supported drivers.

Thanks,
John
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Matt Riedemann]

On 2/23/2017 9:27 PM, Zhenyu Zheng wrote:

BTW, I think this can be done using new placement service, using the
custom resource provider? correct?



This came up briefly on Friday at the PTG but unfortunately jaypipes and 
cdent had already left. You should talk to them about this for more 
details. It's my understanding that at some point you'd have a resource 
provider for the compute node and a resource provider for the cinder 
volume node (same host in this case), and you'd define a type of 
"distance" attribute between them, which would be 0. Then when 
requesting the instance create, you provide the distance qualifier of 0 
meaning you want the server and volume on the same resource provider (if 
possible). The distance attribute would be generic, but for the local 
block device scenario you want it to work with 0 distance.


I'm not exactly sure how this is modeled, but I think it would be via an 
aggregate such that both the compute and volume resource providers are 
in the same aggregate association and the distance=0 is defined on the 
aggregate. If that's the case, I don't see how it makes the setup for 
the operator much less complicated than it is today where there has to 
be a matching AZ for each compute and volume node (which gets unwieldy 
when you start to have several thousand nodes). It would make the user 
experience simpler though because instead of needing to select one out 
of several thousand available AZs, I just create a server with a block 
device and specify distance=0 as a required constraint.


I'm personally not sure how close we are to functionality like that, it 
seems like that would be a ways out to me, i.e. we have a lot of other 
work to do before we get to that point.


--

Thanks,

Matt Riedemann

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Zhenyu Zheng]

BTW, I think this can be done using new placement service, using the custom
resource provider? correct?

On Fri, Feb 24, 2017 at 10:18 AM, Zhenyu Zheng 
wrote:

> Matt,
>
> Thanks for the information, I will check that; But still I think the user
> demand here is to use local disk from
> compute node as block device, as the data can be remained if the old vm
> got deleted, and we can start a
> new one with the data and having the performance they wanted.
>
> Kevin Zheng
>
> On Fri, Feb 24, 2017 at 4:06 AM, Matt Riedemann <
> mrie...@linux.vnet.ibm.com> wrote:
>
>> On 9/26/2016 9:21 PM, Zhenyu Zheng wrote:
>>
>>> Hi,
>>>
>>> Thanks for the reply, actually approach one is not we are looking for,
>>> our demands is to attach the real physical volume from compute node to
>>> VMs,
>>> by this way we can achieve the performance we need for usecases such as
>>> big data, this can be done by cinder using BlockDeviceDriver, it is quite
>>> different from the approach one you mentioned. The only problem now is
>>> that we cannot practially ensure the compute resource located on the same
>>> host with the volume, as Matt mentioned above, currently we have to
>>> arrange 1:1 AZ in Cinder and Nova to do this and it is not practical in
>>> commercial
>>> deployments.
>>>
>>> Thanks.
>>>
>>>
>> Kevin,
>>
>> Is the issue because you can't use ephemeral local disks (it must be a
>> persistent boot from volume)?
>>
>> Have you looked at using the LVM image backend for local storage in Nova?
>> I thought cfriesen said once that windriver is doing high performance
>> config using local LVM in nova.
>>
>> --
>>
>> Thanks,
>>
>> Matt Riedemann
>>
>>
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Zhenyu Zheng]

Matt,

Thanks for the information, I will check that; But still I think the user
demand here is to use local disk from
compute node as block device, as the data can be remained if the old vm got
deleted, and we can start a
new one with the data and having the performance they wanted.

Kevin Zheng

On Fri, Feb 24, 2017 at 4:06 AM, Matt Riedemann 
wrote:

> On 9/26/2016 9:21 PM, Zhenyu Zheng wrote:
>
>> Hi,
>>
>> Thanks for the reply, actually approach one is not we are looking for,
>> our demands is to attach the real physical volume from compute node to
>> VMs,
>> by this way we can achieve the performance we need for usecases such as
>> big data, this can be done by cinder using BlockDeviceDriver, it is quite
>> different from the approach one you mentioned. The only problem now is
>> that we cannot practially ensure the compute resource located on the same
>> host with the volume, as Matt mentioned above, currently we have to
>> arrange 1:1 AZ in Cinder and Nova to do this and it is not practical in
>> commercial
>> deployments.
>>
>> Thanks.
>>
>>
> Kevin,
>
> Is the issue because you can't use ephemeral local disks (it must be a
> persistent boot from volume)?
>
> Have you looked at using the LVM image backend for local storage in Nova?
> I thought cfriesen said once that windriver is doing high performance
> config using local LVM in nova.
>
> --
>
> Thanks,
>
> Matt Riedemann
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Matt Riedemann]

On 9/26/2016 9:21 PM, Zhenyu Zheng wrote:

Hi,

Thanks for the reply, actually approach one is not we are looking for,
our demands is to attach the real physical volume from compute node to VMs,
by this way we can achieve the performance we need for usecases such as
big data, this can be done by cinder using BlockDeviceDriver, it is quite
different from the approach one you mentioned. The only problem now is
that we cannot practially ensure the compute resource located on the same
host with the volume, as Matt mentioned above, currently we have to
arrange 1:1 AZ in Cinder and Nova to do this and it is not practical in
commercial
deployments.

Thanks.



Kevin,

Is the issue because you can't use ephemeral local disks (it must be a 
persistent boot from volume)?


Have you looked at using the LVM image backend for local storage in 
Nova? I thought cfriesen said once that windriver is doing high 
performance config using local LVM in nova.


--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Erlon Cruz]

Kevin,

Now that you had a first feedback about the idea, as Jay said, the next
steps is to write a blueprint/spec so other folks in Cinder can better
understand/suggest/vote on what you are proposing.


Erlon

On Sat, Oct 8, 2016 at 12:14 AM, Zhenyu Zheng 
wrote:

> So do we like the idea of "volume based scheduling?"
>
> On Tue, Sep 27, 2016 at 11:39 AM, Joshua Harlow 
> wrote:
>
>> Huang Zhiteng wrote:
>>
>>>
>>>
>>> On Tue, Sep 27, 2016 at 12:00 AM, Joshua Harlow >> > wrote:
>>>
>>> Huang Zhiteng wrote:
>>>
>>>
>>> On Mon, Sep 26, 2016 at 12:05 PM, Joshua Harlow
>>> 
>>> >>
>>>
>>> wrote:
>>>
>>>  Huang Zhiteng wrote:
>>>
>>>  In eBay, we did some inhouse change to Nova so that our
>>> big data
>>>  type of
>>>  use case can have physical disks as ephemeral disk for
>>> this type of
>>>  flavors.  It works well so far.   My 2 cents.
>>>
>>>
>>>  Is there a published patch (or patchset) anywhere that
>>> people can
>>>  look at for said in-house changes?
>>>
>>>
>>> Unfortunately no, but I think we can publish it if there are
>>> enough
>>> interests.  However, I don't think that can be easily adopted
>>> onto
>>> upstream Nova since it depends on other in-house changes we've
>>> done to Nova.
>>>
>>>
>>> Is there any blog, or other that explains the full bunch of changes
>>> that ebay has done (u got me curious)?
>>>
>>> The nice thing about OSS is that if u just get the patchsets out
>>> (even to github or somewhere), those patches may trigger things to
>>> change to match your usecase better just by the nature of people
>>> being able to read them; but if they are never put out there, then
>>> well ya, it's a little hard to get anything to change.
>>>
>>>
>>> Anything stopping a full release of all in-house changes?
>>>
>>> Even if they are not 'super great quality' it really doesn't matter
>>> :)
>>>
>>> Apology for sidetracking the topic a bit.  While we encourage our
>>> engineers to embrace community and open source, I think we didn't do a
>>> good job to actually emphasize that. 'Time To Market' is another factor,
>>> usually a feature requirement becomes deployed service in 2,3 sprint
>>> (4~6 weeks), but you know how much can be done in same amount of time in
>>> community, especially with Nova. :)
>>>
>>
>> Ya, sorry for side-tracking,
>>
>> Overall yes I do know getting changes done in upstream is not a 4-6 week
>> process (though maybe someday it could be). In general I don't want to turn
>> this into a rant, and thankfully I think there is a decent LWN article
>> about this kind of situation already. You might like it :)
>>
>> https://lwn.net/Articles/647524/ (replace embedded linux/kernel in this
>> with openstack and imho it's equally useful/relevant).
>>
>>
>> -Josh
>>
>>
>>
>>
>>
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Zhenyu Zheng]

So do we like the idea of "volume based scheduling?"

On Tue, Sep 27, 2016 at 11:39 AM, Joshua Harlow 
wrote:

> Huang Zhiteng wrote:
>
>>
>>
>> On Tue, Sep 27, 2016 at 12:00 AM, Joshua Harlow > > wrote:
>>
>> Huang Zhiteng wrote:
>>
>>
>> On Mon, Sep 26, 2016 at 12:05 PM, Joshua Harlow
>> 
>> >>
>>
>> wrote:
>>
>>  Huang Zhiteng wrote:
>>
>>  In eBay, we did some inhouse change to Nova so that our
>> big data
>>  type of
>>  use case can have physical disks as ephemeral disk for
>> this type of
>>  flavors.  It works well so far.   My 2 cents.
>>
>>
>>  Is there a published patch (or patchset) anywhere that
>> people can
>>  look at for said in-house changes?
>>
>>
>> Unfortunately no, but I think we can publish it if there are
>> enough
>> interests.  However, I don't think that can be easily adopted onto
>> upstream Nova since it depends on other in-house changes we've
>> done to Nova.
>>
>>
>> Is there any blog, or other that explains the full bunch of changes
>> that ebay has done (u got me curious)?
>>
>> The nice thing about OSS is that if u just get the patchsets out
>> (even to github or somewhere), those patches may trigger things to
>> change to match your usecase better just by the nature of people
>> being able to read them; but if they are never put out there, then
>> well ya, it's a little hard to get anything to change.
>>
>>
>> Anything stopping a full release of all in-house changes?
>>
>> Even if they are not 'super great quality' it really doesn't matter :)
>>
>> Apology for sidetracking the topic a bit.  While we encourage our
>> engineers to embrace community and open source, I think we didn't do a
>> good job to actually emphasize that. 'Time To Market' is another factor,
>> usually a feature requirement becomes deployed service in 2,3 sprint
>> (4~6 weeks), but you know how much can be done in same amount of time in
>> community, especially with Nova. :)
>>
>
> Ya, sorry for side-tracking,
>
> Overall yes I do know getting changes done in upstream is not a 4-6 week
> process (though maybe someday it could be). In general I don't want to turn
> this into a rant, and thankfully I think there is a decent LWN article
> about this kind of situation already. You might like it :)
>
> https://lwn.net/Articles/647524/ (replace embedded linux/kernel in this
> with openstack and imho it's equally useful/relevant).
>
>
> -Josh
>
>
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Joshua Harlow]

Huang Zhiteng wrote:



On Tue, Sep 27, 2016 at 12:00 AM, Joshua Harlow > wrote:

Huang Zhiteng wrote:


On Mon, Sep 26, 2016 at 12:05 PM, Joshua Harlow

>>
wrote:

 Huang Zhiteng wrote:

 In eBay, we did some inhouse change to Nova so that our
big data
 type of
 use case can have physical disks as ephemeral disk for
this type of
 flavors.  It works well so far.   My 2 cents.


 Is there a published patch (or patchset) anywhere that
people can
 look at for said in-house changes?


Unfortunately no, but I think we can publish it if there are enough
interests.  However, I don't think that can be easily adopted onto
upstream Nova since it depends on other in-house changes we've
done to Nova.


Is there any blog, or other that explains the full bunch of changes
that ebay has done (u got me curious)?

The nice thing about OSS is that if u just get the patchsets out
(even to github or somewhere), those patches may trigger things to
change to match your usecase better just by the nature of people
being able to read them; but if they are never put out there, then
well ya, it's a little hard to get anything to change.


Anything stopping a full release of all in-house changes?

Even if they are not 'super great quality' it really doesn't matter :)

Apology for sidetracking the topic a bit.  While we encourage our
engineers to embrace community and open source, I think we didn't do a
good job to actually emphasize that. 'Time To Market' is another factor,
usually a feature requirement becomes deployed service in 2,3 sprint
(4~6 weeks), but you know how much can be done in same amount of time in
community, especially with Nova. :)


Ya, sorry for side-tracking,

Overall yes I do know getting changes done in upstream is not a 4-6 week 
process (though maybe someday it could be). In general I don't want to 
turn this into a rant, and thankfully I think there is a decent LWN 
article about this kind of situation already. You might like it :)


https://lwn.net/Articles/647524/ (replace embedded linux/kernel in this 
with openstack and imho it's equally useful/relevant).


-Josh






__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Huang Zhiteng]

On Tue, Sep 27, 2016 at 9:21 AM, Zhenyu Zheng 
wrote:

> Hi,
>
> Thanks for the reply, actually approach one is not we are looking for, our
> demands is to attach the real physical volume from compute node to VMs,
> by this way we can achieve the performance we need for usecases such as
> big data, this can be done by cinder using BlockDeviceDriver, it is quite
> different from the approach one you mentioned. The only problem now is
> that we cannot practially ensure the compute resource located on the same
> host with the volume, as Matt mentioned above, currently we have to
> arrange 1:1 AZ in Cinder and Nova to do this and it is not practical in
> commercial
> deployments.
>
That's exactly what I suggested, you don't need Cinder to 'passthrough' a
physical drive to a VM.  Do it with Nova (with some change) is much easier
than trying to coordinate between two services.

>
> Thanks.
>
> On Mon, Sep 26, 2016 at 9:48 PM, Erlon Cruz  wrote:
>
>>
>>
>> On Fri, Sep 23, 2016 at 10:19 PM, Zhenyu Zheng > > wrote:
>>
>>> Hi,
>>>
>>> Thanks all for the information, as for the filter Erlon(
>>> InstanceLocalityFilter) mentioned, this only solves a part of the
>>> problem,
>>> we can create new volumes for existing instances using this filter and
>>> then attach to it, but the root volume still cannot
>>> be guranteed to be on the same host as the compute resource, right?
>>>
>>>
>> You have two options to use a disk in the same node as the instance.
>> 1 - The easiest, just don't use Cinder volumes. When you create an
>> instance from an image, the default behavior in Nova, is to create the root
>> disk in the local host (/var/lib/nova/instances). This have the advantage
>> that Nova will cache the image locally and will avoid the need of copying
>> the image over the wire (or having to configure image caching in Cinder).
>>
>> 2 - Use Cinder volumes as root disk. Nova will somehow have to pass the
>> hints to the scheduler so it properly can use the InstanceLocalityFilter.
>> If you place this in Nova, and make sure that all requests have the proper
>> hint, then the volumes created will be scheduled and the host.
>>
>> Is there any reason why you can't use the first approach?
>>
>>
>>
>>
>>> The idea here is that all the volumes uses local disks.
>>> I was wondering if we already have such a plan after the Resource
>>> Provider structure has accomplished?
>>>
>>> Thanks
>>>
>>> On Sat, Sep 24, 2016 at 2:05 AM, Erlon Cruz  wrote:
>>>
 Not sure exactly what you mean, but in Cinder using the
 InstanceLocalityFilter[1], you can  schedule a volume to the same compute
 node the instance is located. Is this what you need?

 [1] http://docs.openstack.org/developer/cinder/scheduler-fil
 ters.html#instancelocalityfilter

 On Fri, Sep 23, 2016 at 12:19 PM, Jay S. Bryant <
 jsbry...@electronicjungle.net> wrote:

> Kevin,
>
> This is functionality that has been requested in the past but has
> never been implemented.
>
> The best way to proceed would likely be to propose a blueprint/spec
> for this and start working this through that.
>
> -Jay
>
>
> On 09/23/2016 02:51 AM, Zhenyu Zheng wrote:
>
> Hi Novaers and Cinders:
>
> Quite often application requirements would demand using locally
> attached disks (or direct attached disks) for OpenStack compute instances.
> One such example is running virtual hadoop clusters via OpenStack.
>
> We can now achieve this by using BlockDeviceDriver as Cinder driver
> and using AZ in Nova and Cinder, illustrated in[1], which is not very
> feasible in large scale production deployment.
>
> Now that Nova is working on resource provider trying to build an
> generic-resource-pool, is it possible to perform "volume-based-scheduling"
> to build instances according to volume? As this could be much easier to
> build instances like mentioned above.
>
> Or do we have any other ways of doing this?
>
> References:
> [1] http://cloudgeekz.com/71/how-to-setup-openstack-to-use-l
> ocal-disks-for-instances.html
>
> Thanks,
>
> Kevin Zheng
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.op
> enstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>

 

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Huang Zhiteng]

On Tue, Sep 27, 2016 at 12:00 AM, Joshua Harlow 
wrote:

> Huang Zhiteng wrote:
>
>>
>> On Mon, Sep 26, 2016 at 12:05 PM, Joshua Harlow > > wrote:
>>
>> Huang Zhiteng wrote:
>>
>> In eBay, we did some inhouse change to Nova so that our big data
>> type of
>> use case can have physical disks as ephemeral disk for this type
>> of
>> flavors.  It works well so far.   My 2 cents.
>>
>>
>> Is there a published patch (or patchset) anywhere that people can
>> look at for said in-house changes?
>>
>>
>> Unfortunately no, but I think we can publish it if there are enough
>> interests.  However, I don't think that can be easily adopted onto
>> upstream Nova since it depends on other in-house changes we've done to
>> Nova.
>>
>>
> Is there any blog, or other that explains the full bunch of changes that
> ebay has done (u got me curious)?
>
> The nice thing about OSS is that if u just get the patchsets out (even to
> github or somewhere), those patches may trigger things to change to match
> your usecase better just by the nature of people being able to read them;
> but if they are never put out there, then well ya, it's a little hard to
> get anything to change.


> Anything stopping a full release of all in-house changes?
>
> Even if they are not 'super great quality' it really doesn't matter :)

Apology for sidetracking the topic a bit.  While we encourage our engineers
to embrace community and open source, I think we didn't do a good job to
actually emphasize that.  'Time To Market' is another factor, usually a
feature requirement becomes deployed service in 2,3 sprint (4~6 weeks), but
you know how much can be done in same amount of time in community,
especially with Nova. :)

>
>
> -Josh
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Regards
Huang Zhiteng
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Zhenyu Zheng]

Hi,

Thanks for the reply, actually approach one is not we are looking for, our
demands is to attach the real physical volume from compute node to VMs,
by this way we can achieve the performance we need for usecases such as big
data, this can be done by cinder using BlockDeviceDriver, it is quite
different from the approach one you mentioned. The only problem now is that
we cannot practially ensure the compute resource located on the same
host with the volume, as Matt mentioned above, currently we have to arrange
1:1 AZ in Cinder and Nova to do this and it is not practical in commercial
deployments.

Thanks.

On Mon, Sep 26, 2016 at 9:48 PM, Erlon Cruz  wrote:

>
>
> On Fri, Sep 23, 2016 at 10:19 PM, Zhenyu Zheng 
> wrote:
>
>> Hi,
>>
>> Thanks all for the information, as for the filter Erlon(
>> InstanceLocalityFilter) mentioned, this only solves a part of the
>> problem,
>> we can create new volumes for existing instances using this filter and
>> then attach to it, but the root volume still cannot
>> be guranteed to be on the same host as the compute resource, right?
>>
>>
> You have two options to use a disk in the same node as the instance.
> 1 - The easiest, just don't use Cinder volumes. When you create an
> instance from an image, the default behavior in Nova, is to create the root
> disk in the local host (/var/lib/nova/instances). This have the advantage
> that Nova will cache the image locally and will avoid the need of copying
> the image over the wire (or having to configure image caching in Cinder).
>
> 2 - Use Cinder volumes as root disk. Nova will somehow have to pass the
> hints to the scheduler so it properly can use the InstanceLocalityFilter.
> If you place this in Nova, and make sure that all requests have the proper
> hint, then the volumes created will be scheduled and the host.
>
> Is there any reason why you can't use the first approach?
>
>
>
>
>> The idea here is that all the volumes uses local disks.
>> I was wondering if we already have such a plan after the Resource
>> Provider structure has accomplished?
>>
>> Thanks
>>
>> On Sat, Sep 24, 2016 at 2:05 AM, Erlon Cruz  wrote:
>>
>>> Not sure exactly what you mean, but in Cinder using the
>>> InstanceLocalityFilter[1], you can  schedule a volume to the same compute
>>> node the instance is located. Is this what you need?
>>>
>>> [1] http://docs.openstack.org/developer/cinder/scheduler-fil
>>> ters.html#instancelocalityfilter
>>>
>>> On Fri, Sep 23, 2016 at 12:19 PM, Jay S. Bryant <
>>> jsbry...@electronicjungle.net> wrote:
>>>
 Kevin,

 This is functionality that has been requested in the past but has never
 been implemented.

 The best way to proceed would likely be to propose a blueprint/spec for
 this and start working this through that.

 -Jay


 On 09/23/2016 02:51 AM, Zhenyu Zheng wrote:

 Hi Novaers and Cinders:

 Quite often application requirements would demand using locally
 attached disks (or direct attached disks) for OpenStack compute instances.
 One such example is running virtual hadoop clusters via OpenStack.

 We can now achieve this by using BlockDeviceDriver as Cinder driver and
 using AZ in Nova and Cinder, illustrated in[1], which is not very feasible
 in large scale production deployment.

 Now that Nova is working on resource provider trying to build an
 generic-resource-pool, is it possible to perform "volume-based-scheduling"
 to build instances according to volume? As this could be much easier to
 build instances like mentioned above.

 Or do we have any other ways of doing this?

 References:
 [1] http://cloudgeekz.com/71/how-to-setup-openstack-to-use-l
 ocal-disks-for-instances.html

 Thanks,

 Kevin Zheng


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: 
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.op
 enstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


>>>
>>> 
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.op
>>> enstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage 

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Joshua Harlow]

Huang Zhiteng wrote:


On Mon, Sep 26, 2016 at 12:05 PM, Joshua Harlow > wrote:

Huang Zhiteng wrote:

In eBay, we did some inhouse change to Nova so that our big data
type of
use case can have physical disks as ephemeral disk for this type of
flavors.  It works well so far.   My 2 cents.


Is there a published patch (or patchset) anywhere that people can
look at for said in-house changes?


Unfortunately no, but I think we can publish it if there are enough
interests.  However, I don't think that can be easily adopted onto
upstream Nova since it depends on other in-house changes we've done to Nova.



Is there any blog, or other that explains the full bunch of changes that 
ebay has done (u got me curious)?


The nice thing about OSS is that if u just get the patchsets out (even 
to github or somewhere), those patches may trigger things to change to 
match your usecase better just by the nature of people being able to 
read them; but if they are never put out there, then well ya, it's a 
little hard to get anything to change.


Anything stopping a full release of all in-house changes?

Even if they are not 'super great quality' it really doesn't matter :)

-Josh

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Erlon Cruz]

On Fri, Sep 23, 2016 at 10:19 PM, Zhenyu Zheng 
wrote:

> Hi,
>
> Thanks all for the information, as for the filter Erlon(
> InstanceLocalityFilter) mentioned, this only solves a part of the problem,
> we can create new volumes for existing instances using this filter and
> then attach to it, but the root volume still cannot
> be guranteed to be on the same host as the compute resource, right?
>
>
You have two options to use a disk in the same node as the instance.
1 - The easiest, just don't use Cinder volumes. When you create an instance
from an image, the default behavior in Nova, is to create the root disk in
the local host (/var/lib/nova/instances). This have the advantage that Nova
will cache the image locally and will avoid the need of copying the image
over the wire (or having to configure image caching in Cinder).

2 - Use Cinder volumes as root disk. Nova will somehow have to pass the
hints to the scheduler so it properly can use the InstanceLocalityFilter.
If you place this in Nova, and make sure that all requests have the proper
hint, then the volumes created will be scheduled and the host.

Is there any reason why you can't use the first approach?




> The idea here is that all the volumes uses local disks.
> I was wondering if we already have such a plan after the Resource Provider
> structure has accomplished?
>
> Thanks
>
> On Sat, Sep 24, 2016 at 2:05 AM, Erlon Cruz  wrote:
>
>> Not sure exactly what you mean, but in Cinder using the
>> InstanceLocalityFilter[1], you can  schedule a volume to the same compute
>> node the instance is located. Is this what you need?
>>
>> [1] http://docs.openstack.org/developer/cinder/scheduler-fil
>> ters.html#instancelocalityfilter
>>
>> On Fri, Sep 23, 2016 at 12:19 PM, Jay S. Bryant <
>> jsbry...@electronicjungle.net> wrote:
>>
>>> Kevin,
>>>
>>> This is functionality that has been requested in the past but has never
>>> been implemented.
>>>
>>> The best way to proceed would likely be to propose a blueprint/spec for
>>> this and start working this through that.
>>>
>>> -Jay
>>>
>>>
>>> On 09/23/2016 02:51 AM, Zhenyu Zheng wrote:
>>>
>>> Hi Novaers and Cinders:
>>>
>>> Quite often application requirements would demand using locally attached
>>> disks (or direct attached disks) for OpenStack compute instances. One such
>>> example is running virtual hadoop clusters via OpenStack.
>>>
>>> We can now achieve this by using BlockDeviceDriver as Cinder driver and
>>> using AZ in Nova and Cinder, illustrated in[1], which is not very feasible
>>> in large scale production deployment.
>>>
>>> Now that Nova is working on resource provider trying to build an
>>> generic-resource-pool, is it possible to perform "volume-based-scheduling"
>>> to build instances according to volume? As this could be much easier to
>>> build instances like mentioned above.
>>>
>>> Or do we have any other ways of doing this?
>>>
>>> References:
>>> [1] http://cloudgeekz.com/71/how-to-setup-openstack-to-use-l
>>> ocal-disks-for-instances.html
>>>
>>> Thanks,
>>>
>>> Kevin Zheng
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: 
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>>
>>> 
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.op
>>> enstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Huang Zhiteng]

On Mon, Sep 26, 2016 at 12:05 PM, Joshua Harlow 
wrote:

> Huang Zhiteng wrote:
>
>> In eBay, we did some inhouse change to Nova so that our big data type of
>> use case can have physical disks as ephemeral disk for this type of
>> flavors.  It works well so far.   My 2 cents.
>>
>
> Is there a published patch (or patchset) anywhere that people can look at
> for said in-house changes?
>

Unfortunately no, but I think we can publish it if there are enough
interests.  However, I don't think that can be easily adopted onto upstream
Nova since it depends on other in-house changes we've done to Nova.

>
> -Josh
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Regards
Huang Zhiteng
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Joshua Harlow]

Huang Zhiteng wrote:

In eBay, we did some inhouse change to Nova so that our big data type of
use case can have physical disks as ephemeral disk for this type of
flavors.  It works well so far.   My 2 cents.


Is there a published patch (or patchset) anywhere that people can look 
at for said in-house changes?


-Josh

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Huang Zhiteng]

Hi Zhenyu and all,

If you look at the problem from a different angle, for example, treating
local disks on hypervisors same resource like GPU/NIC, your requirement
doesn't necessarily need to involve Cinder.  Local disks become a resource
type associated with certain group of hypervisors, scheduling becomes
easier and provisioning is also simpler because it doesn't have to talk to
another service (Cinder) and do coordination between Nova and Cinder
anymore.

In eBay, we did some inhouse change to Nova so that our big data type of
use case can have physical disks as ephemeral disk for this type of
flavors.  It works well so far.   My 2 cents.


On Mon, Sep 26, 2016 at 9:35 AM, Zhenyu Zheng 
wrote:

> Hi Matt,
>
> Yes, we can only do this using 1:1 AZs mapped for each compute node in the
> deployment, which is not very feasible in commercial deployment,
> we can either pass some hints to Cinder(for current code, cinder 
> "InstanceLocalityFilter"
> uses instance uuid as parameter so it will be impossible for
> user to pass it while booting instances)/ add filters or something else to
> Nova when doing Nova scheduling. And maybe we will have new solutions
> after "Generic-resource-pool" is reached?
>
> The implementations may varies, but this could be a reasonable demands?
> right?
>
> Thanks
>
> On Sun, Sep 25, 2016 at 1:02 AM, Matt Riedemann <
> mrie...@linux.vnet.ibm.com> wrote:
>
>> On 9/23/2016 8:19 PM, Zhenyu Zheng wrote:
>>
>>> Hi,
>>>
>>> Thanks all for the information, as for the filter
>>> Erlon(InstanceLocalityFilter) mentioned, this only solves a part of the
>>> problem,
>>> we can create new volumes for existing instances using this filter and
>>> then attach to it, but the root volume still cannot
>>> be guranteed to be on the same host as the compute resource, right?
>>>
>>> The idea here is that all the volumes uses local disks.
>>> I was wondering if we already have such a plan after the Resource
>>> Provider structure has accomplished?
>>>
>>> Thanks
>>>
>>> On Sat, Sep 24, 2016 at 2:05 AM, Erlon Cruz >> > wrote:
>>>
>>> Not sure exactly what you mean, but in Cinder using the
>>> InstanceLocalityFilter[1], you can  schedule a volume to the same
>>> compute node the instance is located. Is this what you need?
>>>
>>> [1] http://docs.openstack.org/developer/cinder/scheduler-filters
>>> .html#instancelocalityfilter
>>> >> s.html#instancelocalityfilter>
>>>
>>> On Fri, Sep 23, 2016 at 12:19 PM, Jay S. Bryant
>>> >> > wrote:
>>>
>>> Kevin,
>>>
>>> This is functionality that has been requested in the past but
>>> has never been implemented.
>>>
>>> The best way to proceed would likely be to propose a
>>> blueprint/spec for this and start working this through that.
>>>
>>> -Jay
>>>
>>>
>>> On 09/23/2016 02:51 AM, Zhenyu Zheng wrote:
>>>
 Hi Novaers and Cinders:

 Quite often application requirements would demand using
 locally attached disks (or direct attached disks) for
 OpenStack compute instances. One such example is running
 virtual hadoop clusters via OpenStack.

 We can now achieve this by using BlockDeviceDriver as Cinder
 driver and using AZ in Nova and Cinder, illustrated in[1],
 which is not very feasible in large scale production deployment.

 Now that Nova is working on resource provider trying to build
 an generic-resource-pool, is it possible to perform
 "volume-based-scheduling" to build instances according to
 volume? As this could be much easier to build instances like
 mentioned above.

 Or do we have any other ways of doing this?

 References:
 [1] http://cloudgeekz.com/71/how-t
 o-setup-openstack-to-use-local-disks-for-instances.html
 

 Thanks,

 Kevin Zheng


 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.op
 enstack.org?subject:unsubscribe
 
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstac
 k-dev
 

>>>
>>>
>>> 
>>> __
>>> OpenStack Development Mailing List (not 

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Zhenyu Zheng]

Hi Matt,

Yes, we can only do this using 1:1 AZs mapped for each compute node in the
deployment, which is not very feasible in commercial deployment,
we can either pass some hints to Cinder(for current code, cinder
"InstanceLocalityFilter"
uses instance uuid as parameter so it will be impossible for
user to pass it while booting instances)/ add filters or something else to
Nova when doing Nova scheduling. And maybe we will have new solutions
after "Generic-resource-pool" is reached?

The implementations may varies, but this could be a reasonable demands?
right?

Thanks

On Sun, Sep 25, 2016 at 1:02 AM, Matt Riedemann 
wrote:

> On 9/23/2016 8:19 PM, Zhenyu Zheng wrote:
>
>> Hi,
>>
>> Thanks all for the information, as for the filter
>> Erlon(InstanceLocalityFilter) mentioned, this only solves a part of the
>> problem,
>> we can create new volumes for existing instances using this filter and
>> then attach to it, but the root volume still cannot
>> be guranteed to be on the same host as the compute resource, right?
>>
>> The idea here is that all the volumes uses local disks.
>> I was wondering if we already have such a plan after the Resource
>> Provider structure has accomplished?
>>
>> Thanks
>>
>> On Sat, Sep 24, 2016 at 2:05 AM, Erlon Cruz > > wrote:
>>
>> Not sure exactly what you mean, but in Cinder using the
>> InstanceLocalityFilter[1], you can  schedule a volume to the same
>> compute node the instance is located. Is this what you need?
>>
>> [1] http://docs.openstack.org/developer/cinder/scheduler-filters
>> .html#instancelocalityfilter
>> > s.html#instancelocalityfilter>
>>
>> On Fri, Sep 23, 2016 at 12:19 PM, Jay S. Bryant
>> > > wrote:
>>
>> Kevin,
>>
>> This is functionality that has been requested in the past but
>> has never been implemented.
>>
>> The best way to proceed would likely be to propose a
>> blueprint/spec for this and start working this through that.
>>
>> -Jay
>>
>>
>> On 09/23/2016 02:51 AM, Zhenyu Zheng wrote:
>>
>>> Hi Novaers and Cinders:
>>>
>>> Quite often application requirements would demand using
>>> locally attached disks (or direct attached disks) for
>>> OpenStack compute instances. One such example is running
>>> virtual hadoop clusters via OpenStack.
>>>
>>> We can now achieve this by using BlockDeviceDriver as Cinder
>>> driver and using AZ in Nova and Cinder, illustrated in[1],
>>> which is not very feasible in large scale production deployment.
>>>
>>> Now that Nova is working on resource provider trying to build
>>> an generic-resource-pool, is it possible to perform
>>> "volume-based-scheduling" to build instances according to
>>> volume? As this could be much easier to build instances like
>>> mentioned above.
>>>
>>> Or do we have any other ways of doing this?
>>>
>>> References:
>>> [1] http://cloudgeekz.com/71/how-to-setup-openstack-to-use-local
>>> -disks-for-instances.html
>>> >> l-disks-for-instances.html>
>>>
>>> Thanks,
>>>
>>> Kevin Zheng
>>>
>>>
>>> 
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.op
>>> enstack.org?subject:unsubscribe
>>> >> subscribe>
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstac
>>> k-dev
>>> >> ck-dev>
>>>
>>
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > subscribe>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
>>
>>
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > >
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
>>
>>
>>
>>
>> 

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Matt Riedemann]

On 9/23/2016 8:19 PM, Zhenyu Zheng wrote:

Hi,

Thanks all for the information, as for the filter
Erlon(InstanceLocalityFilter) mentioned, this only solves a part of the
problem,
we can create new volumes for existing instances using this filter and
then attach to it, but the root volume still cannot
be guranteed to be on the same host as the compute resource, right?

The idea here is that all the volumes uses local disks.
I was wondering if we already have such a plan after the Resource
Provider structure has accomplished?

Thanks

On Sat, Sep 24, 2016 at 2:05 AM, Erlon Cruz > wrote:

Not sure exactly what you mean, but in Cinder using the
InstanceLocalityFilter[1], you can  schedule a volume to the same
compute node the instance is located. Is this what you need?

[1] 
http://docs.openstack.org/developer/cinder/scheduler-filters.html#instancelocalityfilter



On Fri, Sep 23, 2016 at 12:19 PM, Jay S. Bryant
> wrote:

Kevin,

This is functionality that has been requested in the past but
has never been implemented.

The best way to proceed would likely be to propose a
blueprint/spec for this and start working this through that.

-Jay


On 09/23/2016 02:51 AM, Zhenyu Zheng wrote:

Hi Novaers and Cinders:

Quite often application requirements would demand using
locally attached disks (or direct attached disks) for
OpenStack compute instances. One such example is running
virtual hadoop clusters via OpenStack.

We can now achieve this by using BlockDeviceDriver as Cinder
driver and using AZ in Nova and Cinder, illustrated in[1],
which is not very feasible in large scale production deployment.

Now that Nova is working on resource provider trying to build
an generic-resource-pool, is it possible to perform
"volume-based-scheduling" to build instances according to
volume? As this could be much easier to build instances like
mentioned above.

Or do we have any other ways of doing this?

References:
[1] 
http://cloudgeekz.com/71/how-to-setup-openstack-to-use-local-disks-for-instances.html



Thanks,

Kevin Zheng



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev 




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



Are you asking about the scenario where you are creating a server with a 
source_type=blank/image/snapshot bdm and nova creates the volume to 
attach to the server? In that case nova doesn't pass enough information 
to cinder to build the volume on the same host that the server is 
building on. Nova passes an AZ but that would mean you'd need to have 
1:1 AZs mapped for each compute node in the deployment (I think?).


Maybe you're thinking of like nova passing a scheduler hint to cinder 
telling it where to build the volume?


--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Jay S. Bryant]

Erlon,

Thanks for pointing this out.  I didn't know we had this filter!

Jay


On 09/23/2016 01:05 PM, Erlon Cruz wrote:
Not sure exactly what you mean, but in Cinder using the 
InstanceLocalityFilter[1], you can  schedule a volume to the same 
compute node the instance is located. Is this what you need?


[1] 
http://docs.openstack.org/developer/cinder/scheduler-filters.html#instancelocalityfilter


On Fri, Sep 23, 2016 at 12:19 PM, Jay S. Bryant 
> 
wrote:


Kevin,

This is functionality that has been requested in the past but has
never been implemented.

The best way to proceed would likely be to propose a
blueprint/spec for this and start working this through that.

-Jay


On 09/23/2016 02:51 AM, Zhenyu Zheng wrote:

Hi Novaers and Cinders:

Quite often application requirements would demand using locally
attached disks (or direct attached disks) for OpenStack compute
instances. One such example is running virtual hadoop clusters
via OpenStack.

We can now achieve this by using BlockDeviceDriver as Cinder
driver and using AZ in Nova and Cinder, illustrated in[1], which
is not very feasible in large scale production deployment.

Now that Nova is working on resource provider trying to build an
generic-resource-pool, is it possible to perform
"volume-based-scheduling" to build instances according to volume?
As this could be much easier to build instances like mentioned above.

Or do we have any other ways of doing this?

References:
[1]

http://cloudgeekz.com/71/how-to-setup-openstack-to-use-local-disks-for-instances.html



Thanks,

Kevin Zheng


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Zhenyu Zheng]

Hi,

Thanks all for the information, as for the filter Erlon(
InstanceLocalityFilter) mentioned, this only solves a part of the problem,
we can create new volumes for existing instances using this filter and then
attach to it, but the root volume still cannot
be guranteed to be on the same host as the compute resource, right?

The idea here is that all the volumes uses local disks.
I was wondering if we already have such a plan after the Resource Provider
structure has accomplished?

Thanks

On Sat, Sep 24, 2016 at 2:05 AM, Erlon Cruz  wrote:

> Not sure exactly what you mean, but in Cinder using the
> InstanceLocalityFilter[1], you can  schedule a volume to the same compute
> node the instance is located. Is this what you need?
>
> [1] http://docs.openstack.org/developer/cinder/scheduler-filters.html#
> instancelocalityfilter
>
> On Fri, Sep 23, 2016 at 12:19 PM, Jay S. Bryant <
> jsbry...@electronicjungle.net> wrote:
>
>> Kevin,
>>
>> This is functionality that has been requested in the past but has never
>> been implemented.
>>
>> The best way to proceed would likely be to propose a blueprint/spec for
>> this and start working this through that.
>>
>> -Jay
>>
>>
>> On 09/23/2016 02:51 AM, Zhenyu Zheng wrote:
>>
>> Hi Novaers and Cinders:
>>
>> Quite often application requirements would demand using locally attached
>> disks (or direct attached disks) for OpenStack compute instances. One such
>> example is running virtual hadoop clusters via OpenStack.
>>
>> We can now achieve this by using BlockDeviceDriver as Cinder driver and
>> using AZ in Nova and Cinder, illustrated in[1], which is not very feasible
>> in large scale production deployment.
>>
>> Now that Nova is working on resource provider trying to build an
>> generic-resource-pool, is it possible to perform "volume-based-scheduling"
>> to build instances according to volume? As this could be much easier to
>> build instances like mentioned above.
>>
>> Or do we have any other ways of doing this?
>>
>> References:
>> [1] http://cloudgeekz.com/71/how-to-setup-openstack-to-use-l
>> ocal-disks-for-instances.html
>>
>> Thanks,
>>
>> Kevin Zheng
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: 
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Erlon Cruz]

Not sure exactly what you mean, but in Cinder using the
InstanceLocalityFilter[1], you can  schedule a volume to the same compute
node the instance is located. Is this what you need?

[1]
http://docs.openstack.org/developer/cinder/scheduler-filters.html#instancelocalityfilter

On Fri, Sep 23, 2016 at 12:19 PM, Jay S. Bryant <
jsbry...@electronicjungle.net> wrote:

> Kevin,
>
> This is functionality that has been requested in the past but has never
> been implemented.
>
> The best way to proceed would likely be to propose a blueprint/spec for
> this and start working this through that.
>
> -Jay
>
>
> On 09/23/2016 02:51 AM, Zhenyu Zheng wrote:
>
> Hi Novaers and Cinders:
>
> Quite often application requirements would demand using locally attached
> disks (or direct attached disks) for OpenStack compute instances. One such
> example is running virtual hadoop clusters via OpenStack.
>
> We can now achieve this by using BlockDeviceDriver as Cinder driver and
> using AZ in Nova and Cinder, illustrated in[1], which is not very feasible
> in large scale production deployment.
>
> Now that Nova is working on resource provider trying to build an
> generic-resource-pool, is it possible to perform "volume-based-scheduling"
> to build instances according to volume? As this could be much easier to
> build instances like mentioned above.
>
> Or do we have any other ways of doing this?
>
> References:
> [1] http://cloudgeekz.com/71/how-to-setup-openstack-to-use-
> local-disks-for-instances.html
>
> Thanks,
>
> Kevin Zheng
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Jay S. Bryant]

Kevin,

This is functionality that has been requested in the past but has never 
been implemented.


The best way to proceed would likely be to propose a blueprint/spec for 
this and start working this through that.


-Jay


On 09/23/2016 02:51 AM, Zhenyu Zheng wrote:

Hi Novaers and Cinders:

Quite often application requirements would demand using locally 
attached disks (or direct attached disks) for OpenStack compute 
instances. One such example is running virtual hadoop clusters via 
OpenStack.


We can now achieve this by using BlockDeviceDriver as Cinder driver 
and using AZ in Nova and Cinder, illustrated in[1], which is not very 
feasible in large scale production deployment.


Now that Nova is working on resource provider trying to build an 
generic-resource-pool, is it possible to perform 
"volume-based-scheduling" to build instances according to volume? As 
this could be much easier to build instances like mentioned above.


Or do we have any other ways of doing this?

References:
[1] 
http://cloudgeekz.com/71/how-to-setup-openstack-to-use-local-disks-for-instances.html


Thanks,

Kevin Zheng


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova][cinder] Schedule Instances according to Local disk based Volume?

[Zhenyu Zheng]

Hi Novaers and Cinders:

Quite often application requirements would demand using locally attached
disks (or direct attached disks) for OpenStack compute instances. One such
example is running virtual hadoop clusters via OpenStack.

We can now achieve this by using BlockDeviceDriver as Cinder driver and
using AZ in Nova and Cinder, illustrated in[1], which is not very feasible
in large scale production deployment.

Now that Nova is working on resource provider trying to build an
generic-resource-pool, is it possible to perform "volume-based-scheduling"
to build instances according to volume? As this could be much easier to
build instances like mentioned above.

Or do we have any other ways of doing this?

References:
[1]
http://cloudgeekz.com/71/how-to-setup-openstack-to-use-local-disks-for-instances.html

Thanks,

Kevin Zheng
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev