Re: [openstack-dev] [nova] nova cellsv2 and DBs / down cells / quotas

2018-10-25 Thread Dan Smith
> I guess our architecture is pretty unique in a way but I wonder if > other people are also a little scared about the whole all DB servers > need to up to serve API requests? When we started down this path, we acknowledged that this would create a different access pattern which would require ops

Re: [openstack-dev] [nova] Metadata API cross joining "instance_metadata" and "instance_system_metadata"

2018-10-23 Thread Dan Smith
> I tested a code change that essentially reverts > https://review.openstack.org/#/c/276861/1/nova/api/metadata/base.py > > In other words, with this change metadata tables are not fetched by > default in API requests. If I understand correctly, metadata is > fetched in separate queries as the

Re: [openstack-dev] [nova] Metadata API cross joining "instance_metadata" and "instance_system_metadata"

2018-10-22 Thread Dan Smith
> Do you guys see an easy fix here? > > Should I open a bug report? Definitely open a bug. IMHO, we should just make the single-instance load work like the multi ones, where we load the metadata separately if requested. We might be able to get away without sysmeta these days, but we needed it for

Re: [openstack-dev] [nova] Metadata API cross joining "instance_metadata" and "instance_system_metadata"

2018-10-22 Thread Dan Smith
> We haven't been doing this (intentionally) for quite some time, as we > query and fill metadata linearly: > > https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py#L2244 > > and have since 2013 (Havana): > > https://review.openstack.org/#/c/26136/ > > So unless there has been a

Re: [openstack-dev] [nova] Metadata API cross joining "instance_metadata" and "instance_system_metadata"

2018-10-22 Thread Dan Smith
> Of course this is only a problem when instances have a lot of metadata > records. An instance with 50 records in "instance_metadata" and 50 > records in "instance_system_metadata" will fetch 50 x 50 = 2,500 rows > from the database. It's not difficult to see how this can escalate > quickly. This

Re: [openstack-dev] [Openstack-operators] [nova] Supporting force live-migrate and force evacuate with nested allocations

2018-10-10 Thread Dan Smith
>> I disagree on this. I'd rather just do a simple check for >1 >> provider in the allocations on the source and if True, fail hard. >> >> The reverse (going from a non-nested source to a nested destination) >> will hard fail anyway on the destination because the POST >> /allocations won't work

Re: [openstack-dev] [placement] The "intended purpose" of traits

2018-10-01 Thread Dan Smith
>> I still want to use something like "Is capable of RAID5" and/or "Has >> RAID5 already configured" as part of a scheduling and placement >> decision. Being able to have the GET /a_c response filtered down to >> providers with those, ahem, traits is the exact purpose of that operation. > > And

Re: [openstack-dev] [placement] The "intended purpose" of traits

2018-10-01 Thread Dan Smith
> It sounds like you might be saying, "I would rather not see encoded > trait names OR a new key/value primitive; but if the alternative is > ending up with 'a much larger mess', I would accept..." ...which? > > Or is it, "We should not implement a key/value primitive, nor should we > implement

Re: [openstack-dev] [placement] The "intended purpose" of traits

2018-10-01 Thread Dan Smith
I was out when much of this conversation happened, so I'm going to summarize my opinion here. > So from a code perspective _placement_ is completely agnostic to > whether a trait is "PCI_ADDRESS_01_AB_23_CD", "STORAGE_DISK_SSD", or > "JAY_LIKES_CRUNCHIE_BARS". > > However, things which are using

Re: [openstack-dev] Open letter/request to TC candidates (and existing elected officials)

2018-09-12 Thread Dan Smith
> I'm just a bit worried to limit that role to the elected TC members. If > we say "it's the role of the TC to do cross-project PM in OpenStack" > then we artificially limit the number of people who would sign up to do > that kind of work. You mention Ildiko and Lance: they did that line of > work

Re: [openstack-dev] [upgrade] request for pre-upgrade check for db purge

2018-09-11 Thread Dan Smith
> How do people feel about this? It seems pretty straight-forward to > me. If people are generally in favor of this, then the question is > what would be sane defaults - or should we not assume a default and > force operators to opt into this? I dunno, adding something to nova.conf that is only

Re: [openstack-dev] [nova][placement][upgrade][qa] Some upgrade-specific news on extraction

2018-09-07 Thread Dan Smith
> The other obvious thing is the database. The placement repo code as-is > today still has the check for whether or not it should use the > placement database but falls back to using the nova_api database > [5]. So technically you could point the extracted placement at the > same nova_api database

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-09-05 Thread Dan Smith
> I think there was a period in time where the nova_api database was created > where entires would try to get pulled out from the original nova database and > then checking nova_api if it doesn't exist afterwards (or vice versa). One > of the cases that this was done to deal with was for things

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-09-05 Thread Dan Smith
>> Yes, we should definitely trim the placement DB migrations to only >> things relevant to placement. And we can use this opportunity to get >> rid of cruft too and squash all of the placement migrations together >> to start at migration 1 for the placement repo. If anyone can think >> of a

Re: [openstack-dev] [nova][placement] Freezing placement for extraction

2018-08-31 Thread Dan Smith
> If we're going to do the extraction in Stien, which we said we'd do in > Dublin, we need to start that as early as possible to iron out any > deployment bugs in the switch. We can't wait until the 2nd or 3rd > milestone, it would be too risky. I agree that the current extraction plan is highly

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-28 Thread Dan Smith
> Grenade already has it's own "resources db" right? So we can shove > things in there before we upgrade and then verify they are still there > after the upgrade? Yep, I'm working on something right now. We create an instance that survives the upgrade and validate it on the other side. I'll just

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-28 Thread Dan Smith
> Grenade uses devstack so once we have devstack on master installing > (and configuring) placement from the new repo and disable installing > and configuring it from the nova repo, that's the majority of the > change I'd think. > > Grenade will likely need a from-rocky script to move any config

Re: [openstack-dev] [nova] [placement] extraction (technical) update

2018-08-28 Thread Dan Smith
>> 2. We have a stack of changes to zuul jobs that show nova working but >> deploying placement in devstack from the new repo instead of nova's >> repo. This includes the grenade job, ensuring that upgrade works. > > I'm guessing there would need to be changes to Devstack itself, outside > of

Re: [openstack-dev] [oslo] UUID sentinel needs a home

2018-08-23 Thread Dan Smith
> The compromise, using the patch as currently written [1], would entail > adding one line at the top of each test file: > > uuids = uuidsentinel.UUIDSentinels() > > ...as seen (more or less) at [2]. The subtle difference being that this > `uuids` wouldn't share a namespace across the whole

Re: [openstack-dev] [oslo] UUID sentinel needs a home

2018-08-23 Thread Dan Smith
> Do you mean an actual fixture, that would be used like: > > class MyTestCase(testtools.TestCase): > def setUp(self): > self.uuids = self.useFixture(oslofx.UUIDSentinelFixture()).uuids > > def test_foo(self): > do_a_thing_with(self.uuids.foo) > > ? > > That's... okay

Re: [openstack-dev] [nova][cinder][neutron] Cross-cell cold migration

2018-08-23 Thread Dan Smith
> I think Nova should never have to rely on Cinder's hosts/backends > information to do migrations or any other operation. > > In this case even if Nova had that info, it wouldn't be the solution. > Cinder would reject migrations if there's an incompatibility on the > Volume Type (AZ, Referenced

Re: [openstack-dev] [all] [nova] [placement] placement below or beside compute after extraction?

2018-08-20 Thread Dan Smith
>> So my hope is that (in no particular order) Jay Pipes, Eric Fried, >> Takashi Natsume, Tetsuro Nakamura, Matt Riedemann, Andrey Volkov, >> Alex Xu, Balazs Gibizer, Ed Leafe, and any other contributor to >> placement whom I'm forgetting [1] would express their preference on >> what they'd like

Re: [openstack-dev] [all] [nova] [placement] placement below or beside compute after extraction?

2018-08-17 Thread Dan Smith
> The subject of using placement in Cinder has come up, and since then I've had > a > few conversations with people in and outside of that team. I really think > until > placement is its own project outside of the nova team, there will be > resistance > from some to adopt it. I know politics

Re: [openstack-dev] [Nova] A multi-cell instance-list performance test

2018-08-17 Thread Dan Smith
> We have tried out the patch: > https://review.openstack.org/#/c/592698/ > we also applied https://review.openstack.org/#/c/592285/ > > it turns out that we are able to half the overall time consumption, we > did try with different sort key and dirs, the results are similar, we > didn't try out

Re: [openstack-dev] [Nova] A multi-cell instance-list performance test

2018-08-16 Thread Dan Smith
> yes, the DB query was in serial, after some investigation, it seems that we > are unable to perform eventlet.mockey_patch in uWSGI mode, so > Yikun made this fix: > > https://review.openstack.org/#/c/592285/ Cool, good catch :) > > After making this change, we test again, and we got this

Re: [openstack-dev] [nova] increasing the number of allowed volumes attached per instance > 26

2018-06-15 Thread Dan Smith
> I thought we were leaning toward the option where nova itself doesn't > impose a limit, but lets the virt driver decide. > > I would really like NOT to see logic like this in any nova code: > >> if kvm|qemu: >> return 256 >> elif POWER: >> return 4000 >> elif: >>

Re: [openstack-dev] [nova] review runways check-in and feedback

2018-06-14 Thread Dan Smith
> While I have tried to review a few of the runway-slotted efforts, I > have gotten burned out on a number of them. Other runway-slotted > efforts, I simply don't care enough about or once I've seen some of > the code, simply can't bring myself to review it (sorry, just being > honest). I have

Re: [openstack-dev] [nova] increasing the number of allowed volumes attached per instance > 26

2018-06-08 Thread Dan Smith
> Some ideas that have been discussed so far include: FYI, these are already in my order of preference. > A) Selecting a new, higher maximum that still yields reasonable > performance on a single compute host (64 or 128, for example). Pros: > helps prevent the potential for poor performance on a

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

2018-06-01 Thread Dan Smith
> FWIW, I don't have a problem with the virt driver "knowing about > allocations". What I have a problem with is the virt driver *claiming > resources for an instance*. +1000. > That's what the whole placement claims resources things was all about, > and I'm not interested in stepping back to

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

2018-06-01 Thread Dan Smith
> Dan, you are leaving out the parts of my response where I am agreeing > with you and saying that your "Option #2" is probably the things we > should go with. No, what you said was: >> I would vote for Option #2 if it comes down to it. Implying (to me at least) that you still weren't in favor

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

2018-06-01 Thread Dan Smith
> So, you're saying the normal process is to try upgrading the Linux > kernel and associated low-level libs, wait the requisite amount of > time that takes (can be a long time) and just hope that everything > comes back OK? That doesn't sound like any upgrade I've ever seen. I'm saying I think

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

2018-05-31 Thread Dan Smith
> My feeling is that we should not attempt to "migrate" any allocations > or inventories between root or child providers within a compute node, > period. While I agree this is the simplest approach, it does put a lot of responsibility on the operators to do work to sidestep this issue, which

Re: [openstack-dev] [Nova] z/VM introducing a new config driveformat

2018-05-30 Thread Dan Smith
> can I know a use case for this 'live copy metadata or ' the 'only way > to access device tags when hot-attach? my thought is this is one time > thing in cloud-init side either through metatdata service or config > drive and won't be used later? then why I need a live copy? If I do something

Re: [openstack-dev] [StarlingX] StarlingX code followup discussions

2018-05-24 Thread Dan Smith
> For example, I look at your nova fork and it has a "don't allow this > call during an upgrade" decorator on many API calls. Why wasn't that > done upstream? It doesn't seem overly controversial, so it would be > useful to understand the reasoning for that change. Interesting. We have internal

Re: [openstack-dev] [nova] nova-manage cell_v2 map_instances uses invalid UUID as marker in the db

2018-05-10 Thread Dan Smith
> Takashi Natsume writes: > >> In some compute REST APIs, it returns the 'marker' parameter >> in their pagination. >> Then users can specify the 'marker' parameter in the next request. I read this as you saying there was some way that the in-band marker mapping

Re: [openstack-dev] [nova] nova-manage cell_v2 map_instances uses invalid UUID as marker in the db

2018-05-10 Thread Dan Smith
Takashi Natsume writes: > In some compute REST APIs, it returns the 'marker' parameter > in their pagination. > Then users can specify the 'marker' parameter in the next request. How is this possible? The only way we would get the marker is if we either (a) listed

Re: [openstack-dev] [nova] nova-manage cell_v2 map_instances uses invalid UUID as marker in the db

2018-05-10 Thread Dan Smith
> The oslo UUIDField emits a warning if the string used as a field value > does not pass the validation of the uuid.UUID(str(value)) call > [3]. All the offending places are fixed in nova except the nova-manage > cell_v2 map_instances call [1][2]. That call uses markers in the DB > that are not

Re: [openstack-dev] [nova][placement] Trying to summarize bp/glance-image-traits scheduling alternatives for rebuild

2018-05-03 Thread Dan Smith
> I'm late to this thread but I finally went through the replies and my > thought is, we should do a pre-flight check to verify with placement > whether the image traits requested are 1) supported by the compute > host the instance is residing on and 2) coincide with the > already-existing

Re: [openstack-dev] [Nova] z/VM introducing a new config driveformat

2018-04-30 Thread Dan Smith
> According to requirements and comments, now we opened the CI runs with > run_validation = True And according to [1] below, for example, [2] > need the ssh validation passed the test > > And there are a couple of comments need some enhancement on the logs > of CI such as format and legacy

Re: [openstack-dev] [Nova] z/VM introducing a new config driveformat

2018-04-18 Thread Dan Smith
> Having briefly read the cloud-init snippet which was linked earlier in > this thread, the requirement seems to be that the guest exposes the > device as /dev/srX or /dev/cdX. So I guess in order to make this work: > > * You need to tell z/VM to expose the virtual disk as an optical disk > * The

Re: [openstack-dev] [nova] Concern about trusted certificates API change

2018-04-18 Thread Dan Smith
> Maybe it wasn't clear but I'm not advocating that we block the change > until volume-backed instances are supported with trusted certs. I'm > suggesting we add a policy rule which allows deployers to at least > disable it via policy if it's not supported for their cloud. That's fine with me,

Re: [openstack-dev] [Nova] z/VM introducing a new config driveformat

2018-04-18 Thread Dan Smith
> Thanks for the concern and fully under it , the major reason is > cloud-init doesn't have a hook or plugin before it start to read > config drive (ISO disk) z/VM is an old hypervisor and no way to do > something like libvirt to define a ISO format disk in xml definition, > instead, it can define

Re: [openstack-dev] [Nova] z/VM introducing a new config driveformat

2018-04-17 Thread Dan Smith
> I propose that we remove the z/VM driver blueprint from the runway at > this time and place it back into the queue while work on the driver > continues. At a minimum, we need to see z/VM CI running with > [validation]run_validation = True in tempest.conf before we add the > z/VM driver blueprint

Re: [openstack-dev] [Nova][Deployers] Optional, platform specific, dependancies in requirements.txt

2018-04-13 Thread Dan Smith
>> global ironic >> if ironic is None: >> ironic = importutils.import_module('ironicclient') I believe ironic was an early example of a client library we hot-loaded, and I believe at the time we said this was a pattern we were going to follow. Personally, I think this

Re: [openstack-dev] [Nova] z/VM introducing a new config driveformat

2018-04-13 Thread Dan Smith
> for the run_validation=False issue, you are right, because z/VM driver > only support config drive and don't support metadata service ,we made > bad assumption and took wrong action to disabled the whole ssh check, > actually according to [1] , we should only disable >

Re: [openstack-dev] [Nova] z/VM introducing a new config drive format

2018-04-11 Thread Dan Smith
> https://review.openstack.org/#/c/527658 is a z/VM patch which > introduces their support for config drive. They do this by attaching a > tarball to the instance, having pretended in the nova code that it is > an iso9660. This worries me. > > In the past we've been concerned about adding new

Re: [openstack-dev] [nova] [cyborg] Race condition in the Cyborg/Nova flow

2018-03-29 Thread Dan Smith
> ==> Fully dynamic: You can program one region with one function, and > then still program a different region with a different function, etc. Note that this is also the case if you don't have virtualized multi-slot devices. Like, if you had one that only has one region. Consuming it consumes the

Re: [openstack-dev] [nova] Proposing Eric Fried for nova-core

2018-03-27 Thread Dan Smith
> To the existing core team members, please respond with your comments, > +1s, or objections within one week. +1. --Dan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe:

Re: [openstack-dev] [nova] Does Cell v2 support for muti-cell deployment in Pike?

2018-03-23 Thread Dan Smith
> Does Cell v2 support for multi-cell deployment in pike? Is there any > good document about the deployment? In the release notes of Pike: https://docs.openstack.org/releasenotes/nova/pike.html is this under 16.0.0 Prelude: Nova now supports a Cells v2 multi-cell deployment. The default

Re: [openstack-dev] [nova] Rocky spec review day

2018-03-21 Thread Dan Smith
>> And I, for one, wouldn't be offended if we could "officially start >> development" (i.e. focus on patches, start runways, etc.) before the >> mystical but arbitrary spec freeze date. Yeah, I agree. I see runways as an attempt to add pressure to the earlier part of the cycle, where we're

Re: [openstack-dev] [nova] New image backend: StorPool

2018-03-16 Thread Dan Smith
> Can you be more specific about what is limiting you when you use > volume-backed instances? Presumably it's because you're taking a trip over iscsi instead of using the native attachment mechanism for the technology that you're using? If so, that's a valid argument, but it's hard to see the

Re: [openstack-dev] [nova] about rebuild instance booted from volume

2018-03-15 Thread Dan Smith
> Deleting all snapshots would seem dangerous though... > > 1. I want to reset my instance to how it was before > 2. I'll just do a snapshot in case I need any data in the future > 3. rebuild > 4. oops Yep, for sure. I think if there are snapshots, we have to refuse to do te thing. My comment was

Re: [openstack-dev] [nova] about rebuild instance booted from volume

2018-03-15 Thread Dan Smith
> Rather than overload delete_on_termination, could another flag like > delete_on_rebuild be added? Isn't delete_on_termination already the field we want? To me, that field means "nova owns this". If that is true, then we should be able to re-image the volume (in-place is ideal, IMHO) and if not,

Re: [openstack-dev] [Openstack-operators] AggregateMultiTenancyIsolation with multiple (many) projects

2018-03-08 Thread Dan Smith
> 2. Dan Smith mentioned another idea such that we could index the > aggregate metadata keys like filter_tenant_id0, filter_tenant_id1, > ... filter_tenant_idN and then combine those so you have one host > aggregate filter_tenant_id* key per tenant. Yep, and that's what I'v

Re: [openstack-dev] [all] Switching to longer development cycles

2017-12-14 Thread Dan Smith
Ed Leafe writes: > I think you're missing the reality that intermediate releases have > about zero uptake in the real world. We have had milestone releases of > Nova for years, but I challenge you to find me one non-trivial > deployment that uses one of them. To my knowledge,

Re: [openstack-dev] [all] Switching to longer development cycles

2017-12-14 Thread Dan Smith
> In my experience, the longer a patch (or worse, patch series) sits > around, the staler it gets. Others are merging changes, so the > long-lived patch series has to be constantly rebased. This is definitely true. > The 20% developer would be spending a greater proportion of her time > figuring

Re: [openstack-dev] [Nova] Privsep transition state of play

2017-11-05 Thread Dan Smith
> I hope everyone travelling to the Sydney Summit is enjoying jet lag > just as much as I normally do. Revenge is sweet! My big advice is that > caffeine is your friend, and to not lick any of the wildlife. I wasn't planning on licking any of it, but thanks for the warning. > As of just now, all

Re: [openstack-dev] [nova] A way to delete a record in 'host_mappings' table

2017-10-03 Thread Dan Smith
> But the record in 'host_mappings' table of api database is not deleted > (I tried it with nova master 8ca24bf1ff80f39b14726aca22b5cf52603ea5a0). > The cell cannot be deleted if the records for the cell remains in > 'host_mappings' table. > (An error occurs with a message "There are existing

Re: [openstack-dev] [all] Update on Zuul v3 Migration - and what to do about issues

2017-10-03 Thread Dan Smith
> Any update on where we stand on issues now? Because every single patch I > tried to land yesterday was killed by POST_FAILURE in various ways. > Including some really small stuff - https://review.openstack.org/#/c/324720/ Yeah, Nova has only landed eight patches since Thursday. Most of those

Re: [openstack-dev] vGPUs support for Nova - Implementation

2017-10-02 Thread Dan Smith
>> I also think there is value in exposing vGPU in a generic way, irrespective >> of the underlying implementation (whether it is DEMU, mdev, SR-IOV or >> whatever approach Hyper-V/VMWare use). > > That is a big ask. To start with, all GPUs are not created equal, and > various vGPU

[openstack-dev] [nova][stable] attn: No approvals for stable/newton right now

2017-09-29 Thread Dan Smith
Hi all, Due to a zuulv3 bug, we're running an old nova-network test job on master and, as you would expect, failing hard. As a workaround in the meantime, we're[0] going to disable that job entirely so that it runs nowhere. This makes it not run on master (good) but also not run on

Re: [openstack-dev] vGPUs support for Nova - Implementation

2017-09-29 Thread Dan Smith
The concepts of PCI and SR-IOV are, of course, generic They are, although the PowerVM guys have already pointed out that they don't even refer to virtual devices by PCI address and thus anything based on that subsystem isn't going to help them. but I think out of principal we should avoid a

Re: [openstack-dev] vGPUs support for Nova - Implementation

2017-09-28 Thread Dan Smith
In this serie of patches we are generalizing the PCI framework to handle MDEV devices. We arguing it's a lot of patches but most of them are small and the logic behind is basically to make it understand two new fields MDEV_PF and MDEV_VF. That's not really "generalizing the PCI framework to

Re: [openstack-dev] [skip-level-upgrades][fast-forward-upgrades] PTG summary

2017-09-20 Thread Dan Smith
- Modify the `supports-upgrades`[3] and `supports-accessible-upgrades`[4] tags I have yet to look into the formal process around making changes to these tags but I will aim to make a start ASAP. We've previously tried to avoid changing assert tag definitions because we then have to

Re: [openstack-dev] [nova] Proposing Balazs Gibizer for nova-core

2017-08-29 Thread Dan Smith
So to the existing core team members, please respond with a yay/nay and after about a week or so we should have a decision (knowing a few cores are on vacation right now). +1 on the condition that gibi stops finding so many bugs in the stuff I worked on. It's embarrassing. --Dan

Re: [openstack-dev] [oslo.db] [ndb] ndb namespace throughout openstack projects

2017-07-24 Thread Dan Smith
> So, I see your point here, but my concern here is that if we *modify* an > existing schema migration that has already been tested to properly apply > a schema change for MySQL/InnoDB and PostgreSQL with code that is > specific to NDB, we introduce the potential for bugs where users report > that

Re: [openstack-dev] [Openstack-operators] [nova] Does anyone rely on PUT /os-services/disable for non-compute services?

2017-06-13 Thread Dan Smith
Are we allowed to cheat and say auto-disabling non-nova-compute services on startup is a bug and just fix it that way for #2? :) Because (1) it doesn't make sense, as far as we know, and (2) it forces the operator to have to use the API to enable them later just to fix their nova service-list

Re: [openstack-dev] [nova] Does anyone rely on PUT /os-services/disable for non-compute services?

2017-06-13 Thread Dan Smith
So it seems our options are: 1. Allow PUT /os-services/{service_uuid} on any type of service, even if doesn't make sense for non-nova-compute services. 2. Change the behavior of [1] to only disable new "nova-compute" services. Please, #2. Please. --Dan

Re: [openstack-dev] [nova][scheduler][placement] Allocating Complex Resources

2017-06-09 Thread Dan Smith
>> b) a compute node could very well have both local disk and shared >> disk. how would the placement API know which one to pick? This is a >> sorting/weighing decision and thus is something the scheduler is >> responsible for. > I remember having this discussion, and we concluded that a >

Re: [openstack-dev] [nova][scheduler][placement] Allocating Complex Resources

2017-06-09 Thread Dan Smith
>> My current feeling is that we got ourselves into our existing mess >> of ugly, convoluted code when we tried to add these complex >> relationships into the resource tracker and the scheduler. We set >> out to create the placement engine to bring some sanity back to how >> we think about things

Re: [openstack-dev] [upgrades][skip-level][leapfrog] - RFC - Skipping releases when upgrading

2017-05-26 Thread Dan Smith
> I haven't looked at what Keystone is doing, but to the degree they are > using triggers, those triggers would only impact new data operations as > they continue to run into the schema that is straddling between two > versions (e.g. old column/table still exists, data should be synced to > new

Re: [openstack-dev] [upgrades][skip-level][leapfrog] - RFC - Skipping releases when upgrading

2017-05-26 Thread Dan Smith
> As most of the upgrade issues center around database migrations, we > discussed some of the potential pitfalls at length. One approach was to > roll-up all DB migrations into a single repository and run all upgrades > for a given project in one step. Another was to simply have mutliple > python

Re: [openstack-dev] [Nova] [Cells] Stupid question: Cells v2 & AZs

2017-05-24 Thread Dan Smith
> Thanks for answering the base question. So, if AZs are implemented with > haggs, then really, they are truly disjoint from cells (ie, not a subset > of a cell and not a superset of a cell, just unrelated.) Does that > philosophy agree with what you are stating? Correct, aggregates are at the

[openstack-dev] [nova] Boston Forum session recap - cellsv2

2017-05-19 Thread Dan Smith
The etherpad for this session is here [1]. The goal of the session was to get some questions answered that the developers had for operators around the topic of cellsv2. The bulk of the time was spent discussing ways to limit instance scheduling retries in a cellsv2 world where placement

Re: [openstack-dev] [tc] [all] OpenStack moving both too fast and too slow at the same time

2017-05-05 Thread Dan Smith
> +1. ocata's cell v2 stuff added a lot of extra required complexity > with no perceivable benefit to end users. If there was a long term > stable version, then putting it in the non lts release would have > been ok. In absence of lts, I would have recommended the cell v2 > stuff have been done in

Re: [openstack-dev] [nova] [placement] experimenting with extracting placement

2017-03-13 Thread Dan Smith
Interestingly, we just had a meeting about cells and the scheduler, which had quite a bit of overlap on this topic. > That said, as mentioned in the previous email, the priorities for Pike > (and likely Queens) will continue to be, in order: traits, ironic, > shared resource pools, and nested

Re: [openstack-dev] [nova] scaling rabbitmq with cells v2 requires manual database update

2017-02-15 Thread Dan Smith
> The problem is there's no way to update an existing cell's transport_url > via nova-manage. There is: https://review.openstack.org/#/c/431582/ > It appears the only way to get around this is manually deleting the old > cell1 record from the db. No, don't do that :) > I'd like to hear more

[openstack-dev] [nova] Cells meeting on Feb-15 is canceled

2017-02-14 Thread Dan Smith
Hi all, In an epic collision of cosmic coincidences, four of the primary cells meeting attendees have a conflict tomorrow. Since there won't really be anyone around to run (or attend) the meeting, we'll have to cancel again. Next week we will be at the PTG so any meeting will be done there. So,

Re: [openstack-dev] [nova] FYI: cells v1 job is blocked

2017-02-14 Thread Dan Smith
> We have a fix here: Actual link to fix is left as an exercise for the reader? https://review.openstack.org/#/c/433707 --Dan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe:

[openstack-dev] [nova] Cells meeting canceled

2017-02-08 Thread Dan Smith
Hi all, Today's cells meeting is canceled. We're still working on getting ocata out the door, a bunch of normal participants are out today, and not much has transpired for pike just yet. --Dan __ OpenStack Development

Re: [openstack-dev] [nova] [placement] [operators] Optional resource asking or not?

2017-01-25 Thread Dan Smith
> Update on that agreement : I made the necessary modification in the > proposal [1] for not verifying the filters. We now send a request to the > Placement API by introspecting the flavor and we get a list of potential > destinations. Thanks! > When I began doing that modification, I know there

Re: [openstack-dev] [nova] [placement] [operators] Optional resource asking or not?

2017-01-24 Thread Dan Smith
> No. Have administrators set the allocation ratios for the resources they > do not care about exceeding capacity to a very high number. > > If someone previously removed a filter, that doesn't mean that the > resources were not consumed on a host. It merely means the admin was > willing to

[openstack-dev] [nova] No cells meeting next week (Jan 18)

2017-01-11 Thread Dan Smith
Hi all, There will be no cells meeting next week, Jan 18 2017. I'll be in the wilderness and nobody else was brave enough to run it in my absence. Yeah, something like that. --Dan __ OpenStack Development Mailing List (not

Re: [openstack-dev] [oslo][nova]Accessing nullable, not set versioned object field

2016-12-16 Thread Dan Smith
>> NotImplementedError: Cannot load 'nullable_string' in the base class >> >> Is this the correct behavior? > > Yes, that's the expected behaviour. Yes. >> Then what is the expected behavior if the field is also defaulted to >> None? >> >> fields = { >> 'nullable_string':

[openstack-dev] [nova] No cells meetings until 2017

2016-12-14 Thread Dan Smith
Hi all, Given the upcoming holidays, there will not be nova cells meetings for the remainder of the year. That puts the next one at January 4, 2017. --Dan __ OpenStack Development Mailing List (not for usage questions)

Re: [openstack-dev] [Nova] Stepping back

2016-11-22 Thread Dan Smith
> It has been a true pleasure working with you all these past few years > and I'm thankful to have had the opportunity. As I've told people many > times when they ask me what it's like to work on an open source project > like this: working on proprietary software exposes you to smart people > but

[openstack-dev] [nova] 23-Nov Cells meeting cancelled

2016-11-22 Thread Dan Smith
Hi all, Since this week's cells meeting falls on food-coma-day-eve, we're canceling it. Anyone that wants to help move things along could review the following patches: https://review.openstack.org/#/q/topic:bp/cells-sched-staging+project:openstack/nova+status:open

Re: [openstack-dev] [nova] Thanks to those reviewing specs this week

2016-11-17 Thread Dan Smith
> I just wanted to say thanks to everyone reviewing specs this week. I've > seen a lot of non-core newer people to the specs review process chipping > in and helping to review a lot of the specs we're trying to get approved > for Ocata. It can be hard to grind through several specs reviews in a >

Re: [openstack-dev] [nova] Follow up on BCN review cadence discussions

2016-11-08 Thread Dan Smith
> I do imagine, however, that most folks who have been working > on nova for long enough have a list of domain experts in their heads > already. Would actually putting that on paper really hurt? You mean like this? https://wiki.openstack.org/wiki/Nova#Developer_Contacts Those are pretty much

[openstack-dev] [nova] No cellsv2 meeting today

2016-11-02 Thread Dan Smith
Hi all, A bunch of the usual participants cannot attend the CellsV2 meeting today, and the ones that can just discussed it last week face-to-face in Barcelona. So, I'm going to declare it canceled for today for lack of critical mass. --Dan

Re: [openstack-dev] [nova] VIF plugin issue _get_neutron_events

2016-10-08 Thread Dan Smith
> Basically the issue is seen in the following three lines of nova compute > log. For that port even though it received the vif plugging event 2 mins > before it waits for it and blocks and times out > Is there a race condition in the code that basically gets the events to > wait for and the one

Re: [openstack-dev] [nova] Draft Ocata design summit schedule is up

2016-10-06 Thread Dan Smith
> Is there a particular reason we're only retrospecting on placement? I think that we need to have a concrete topic that applied to newton and will apply to ocata in order to be productive. I think there will be specific things we can change in ocata that will have an actual impact on major work

Re: [openstack-dev] [nova] Future of turbo-hipster CI

2016-10-04 Thread Dan Smith
> Having said that, I think Dan Smith came across a fairly large > production DB dataset recently which he was using for testing some > archive changes, maybe Dan will become our new Johannes, but grumpier of > course. :) That's quite an insult to Johannes :) While working on the

Re: [openstack-dev] [nova] ops meetup feedback

2016-09-20 Thread Dan Smith
> The current DB online data upgrade model feels *very opaque* to > ops. They didn't realize the current model Nova was using, and didn't > feel like it was documented anywhere. > ACTION: document the DB data lifecycle better for operators This is on me, so I'll take it. I've just thrown

Re: [openstack-dev] [nova] Next steps for resource providers work

2016-09-02 Thread Dan Smith
> We know: > > * It pretty much does what we intend it to do: allocations are added > and deleted on server create and delete. > * On manipulations like a resize the allocations are not updated > immediately, there is a delay until the heal periodic job does its > thing. We know one more

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

2016-09-01 Thread Dan Smith
> So that is fine. However, correct me if I'm wrong but you're > proposing just that these projects migrate to also use a new service > layer with oslo.versionedobjects, because IIUC Nova/Neutron's > approach is dependent on that area of indirection being present. > Otherwise, if you meant

Re: [openstack-dev] migrate_flavor_data doesn't flavor migrate meta data of VMs spawned during upgrade.

2016-08-31 Thread Dan Smith
> Thanks Dan for your response. While I do run that before I start my > move to liberty, what I see is that it doesn't seem to flavor migrate > meta data for the VMs that are spawned after controller upgrade from > juno to kilo and before all computes upgraded from juno to kilo. The > current work

Re: [openstack-dev] migrate_flavor_data doesn't flavor migrate meta data of VMs spawned during upgrade.

2016-08-31 Thread Dan Smith
> While migrate_flavor_data seem to flavor migrate meta data of the VMs > that were spawned before upgrade procedure, it doesn't seem to flavor > migrate for the VMs that were spawned during the upgrade procedure more > specifically after openstack controller upgrade and before compute > upgrade.

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

2016-08-30 Thread Dan Smith
>> I don't think it's all that ambitious to think we can just use >> tried and tested schema evolution techniques that work for everyone >> else. > > People have been asking me for over a year how to do this, and I have > no easy answer, I'm glad that you do. I would like to see some > examples

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

2016-08-30 Thread Dan Smith
>> Even in the case of projects using versioned objects, it still >> means a SQL layer has to include functionality for both versions of >> a particular schema change which itself is awkward. That's not true. Nova doesn't have multiple models to straddle a particular change. We just... > It's

  1   2   3   4   >