Awesome, that did the trick. Thanks! I think I can work around this bug when I upgrade our production environment now that I know what I'm looking for.
On Wed, May 23, 2012 at 4:11 PM, Alena Prokharchyk < [email protected]> wrote: > On 5/23/12 1:40 PM, "Jason Davis" <[email protected]> wrote: > > > > > > > > >Here you go. Not sure if attachments will work. If that is true then I > >have it available here as well: > >https://www.box.com/s/3ad5f1cae4230eb02580 > > > > > The attachments work. I confirmed that the problem is related to the stale > references to the removed storage pool. > > Here are storage pools that were removed in Jan, 2012: > > mysql> select id, name, removed from storage_pool where removed is not > null; > +-----+---------+---------------------+ > | id | name | removed | > +-----+---------+---------------------+ > | 201 | XCP NFS | 2012-01-20 19:59:06 | > | 204 | Isilon | 2012-01-30 14:02:00 | > +-----+---------+---------------------+ > 2 rows in set (0.00 sec) > > > > And here are the stale records referencing the removed pools: > > mysql> select id, host_id, pool_id from storage_pool_host_ref where > pool_id in (select id from storage_pool where removed is not null); > +----+---------+---------+ > | id | host_id | pool_id | > +----+---------+---------+ > | 4 | 1 | 204 | > | 5 | 2 | 204 | > +----+---------+---------+ > 2 rows in set (0.00 sec) > > > > During listHosts commands the code gets all the "storage_pool_host_ref" > records for the host_id, and based on pool_ids from the records tries to > get storage pool info. For hosts id=1, id=2 if finds Storage pool with > id=204, and this pool is removed. > > To fix the problem on the current setup, execute: > > delete from storage_pool_host_ref where pool_id in (select id from > storage_pool where removed is not null); > > > > This cleanup will be taken care of as a part of DB upgrade to the Burbank > release, I'll file a ticket for that. > > > -Alena. > > > > > > > > > > >On Wed, May 23, 2012 at 3:32 PM, Alena Prokharchyk > ><[email protected]> wrote: > > > >On 5/23/12 12:54 PM, "Jason Davis" <[email protected]> wrote: > > > >>Would it be better to include these dumps on the original bug submission > >>in > >>Jira or replying here in the mailing list? > > > > > > > >Replying here. > > > > > >> > >>On Wed, May 23, 2012 at 2:15 PM, Alena Prokharchyk < > >>[email protected]> wrote: > >> > >>> On 5/23/12 8:55 AM, "Jason Davis" <[email protected]> wrote: > >>> > >>> >I've tried the upgrade in my development environment and this has > >>>largely > >>> >worked other than the fact that the API command "listHosts" seems to > >>>now > >>> >be > >>> >broken. I've also tried doing a clean install of 3.0.2 and then > >>>restoring > >>> >a > >>> >backup copy of my MySQL DB of the 2.2.14-3.0.2 upgrade and still > >>>encounter > >>> >the same issue. > >>> > > >>> > > >>> >I've opened a bug report > >>>here:http://bugs.cloudstack.org/browse/CS-14846 > >>> > > >>> >Just curious if anyone else has tried this and have had this work > >>> >successfully. > >>> > > >>> > >>> > >>> > >>> The bug indicates that there are some references in > >>> "cloud.storage_pool_host_ref" table to the removed Local Storage > >>>instance > >>> ("cloud.storage_pool" table). The references were supposed to be > >>>removed > >>> along with the storage removal, but due to some bug they were left > >>>around. > >>> > >>> To debug the problem, I need the mysql db dumps of following cloudStack > >>> tables (2 versions - before and after upgrade): > >>> > >>> * host > >>> * storage_pool_host_ref > >>> * storage_pool; > >>> > >>> > >>> Thanks, > >>> Alena. > >>> > >>> > >>> > >>> > >>> > >> > > > > > > > > > > > > > > > > > > > > > > >
