[
https://issues.apache.org/jira/browse/CLOUDSTACK-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715341#comment-13715341
]
Min Chen commented on CLOUDSTACK-3379:
--------------------------------------
Attach our email discussion, reply from Sanjeev,
Hi Min,
I tried the correct steps mentioned below. Here is my objervations:
1.For zone1 management server downloaded the routing template to S3 (Expected
since no SSVM for the first time)
2.For zone2, SSVM in zone1 copied vmware routing template to S3 . Is this the
expected behavior ?
3.CS tried to bring up system vms with proper routing templates in each zone.
Did not chose wrong template. No bug here J
But in my view CLOUDSTACK-3379 is still a valid bug. The situation mentioned in
the bug might arise in the following condition:
1.Created zone1 with S3 secondary and NFS cache store
2.After some point of time assume that user deleted both S3 storage and Cache
storage from CS (Assue no VMs are running including system vms)
3.Created zone2 with S3 and NFS Cache and enabled the zone.
Now CS will try to spin up SSVMs in zone2 but looks for cache store in zone1
and will hit this bug.
> [Object_Store_Refactor] Spinning system vms failed with
> CloudRuntimeException: No cache DataStore in scope id 1 type ZONE
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-3379
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3379
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Management Server
> Affects Versions: 4.2.0
> Environment: Latest build from master branch
> Reporter: Sanjeev N
> Assignee: Min Chen
> Priority: Critical
> Fix For: 4.2.0
>
> Attachments: management-server.rar
>
>
> [Object_Store_Refactor] Spinning system vms failed with
> CloudRuntimeException: No cache DataStore in scope id 1 type ZONE
> System vms failed to come up when there are two zones in the system and only
> one Cache storage.
> Steps to Reproduce:
> ================
> 1.Bring up CS with latest master build
> 2.Add two zones say zone1 with xen and zone2 with vmware clusters
> 3.Add S3 storage and Cache Storage from zone2. Does not specify any cache
> storage in zone1
> 4.Enable both the zones
> Observations:
> ===========
> 1.After step3 vmware system template was successfully download to S3 storage.
> 2,After step4 CS tried to bring up system vms using the esx routing template
> but failed because there is no cache storage in zone1.
> mysql> select * from image_store\G;
> *************************** 1. row ***************************
> id: 6
> name: objectstore
> image_provider_name: S3
> protocol: http
> url: NULL
> data_center_id: NULL
> scope: REGION
> role: Image
> uuid: a62707cd-8cfc-490d-81c6-dfc8d706648f
> parent: NULL
> created: 2013-07-05 15:15:54
> removed: NULL
> total_size: NULL
> used_bytes: NULL
> *************************** 2. row ***************************
> id: 7
> name: nfs://10.147.28.7/export/home/sanjeev/sec_esx_os
> image_provider_name: NFS
> protocol: nfs
> url: nfs://10.147.28.7/export/home/sanjeev/sec_esx_os
> data_center_id: 2
> scope: ZONE
> role: ImageCache
> uuid: 30d3d201-d285-4ce5-a909-598c0abb8cf3
> parent: NULL
> created: 2013-07-05 15:15:54
> removed: NULL
> total_size: NULL
> used_bytes: NULL
> 2 rows in set (0.01 sec)
> ERROR:
> No query specified
> mysql> select * from data_center;
> +----+--------------+--------------------------------------+-------------+---------------+------+---------------+---------------+---------+---------+--------------------+-------------+--------------------+--------+-----------+-------------+---------------+------------------+-------------------+---------------+---------------+---------------+-------------------+------------------+--------------------------------------+---------------------------+--------------------------+---------+-------+---------+-------------+--------------+----------+----------+
> | id | name | uuid | description |
> dns1 | dns2 | internal_dns1 | internal_dns2 | gateway | netmask |
> router_mac_address | mac_address | guest_network_cidr | domain | domain_id |
> networktype | dns_provider | gateway_provider | firewall_provider |
> dhcp_provider | lb_provider | vpn_provider | userdata_provider |
> allocation_state | zone_token |
> is_security_group_enabled | is_local_storage_enabled | removed | owner |
> created | lastUpdated | engine_state | ip6_dns1 | ip6_dns2 |
> +----+--------------+--------------------------------------+-------------+---------------+------+---------------+---------------+---------+---------+--------------------+-------------+--------------------+--------+-----------+-------------+---------------+------------------+-------------------+---------------+---------------+---------------+-------------------+------------------+--------------------------------------+---------------------------+--------------------------+---------+-------+---------+-------------+--------------+----------+----------+
> | 1 | zone1 | 2b76f2bc-a859-4253-a7de-cc7d105eb82a | NULL |
> 10.103.128.16 | NULL | 10.103.128.16 | NULL | NULL | NULL |
> 02:00:00:00:00:01 | 21 | 10.1.1.0/24 | NULL | NULL |
> Advanced | VirtualRouter | VirtualRouter | VirtualRouter |
> VirtualRouter | VirtualRouter | VirtualRouter | VirtualRouter | Enabled
> | 894a4b6d-9721-3bdd-821c-f4bb50c9bb70 | 0 |
> 0 | NULL | NULL | NULL | NULL | Disabled
> | NULL | NULL |
> | 2 | zone2-vmware | 1c0f8682-73fa-42b1-a49a-a160fb688b60 | NULL |
> 10.103.128.16 | NULL | 10.103.128.16 | NULL | NULL | NULL |
> 02:00:00:00:00:01 | 19 | 10.1.1.0/24 | NULL | NULL |
> Advanced | VirtualRouter | VirtualRouter | VirtualRouter |
> VirtualRouter | VirtualRouter | VirtualRouter | VirtualRouter | Enabled
> | 21e6f09b-2b7e-3ad4-8546-0c40394d9b39 | 0 |
> 0 | NULL | NULL | NULL | NULL | Disabled
> | NULL | NULL |
> +----+--------------+--------------------------------------+-------------+---------------+------+---------------+---------------+---------+---------+--------------------+-------------+--------------------+--------+-----------+-------------+---------------+------------------+-------------------+---------------+---------------+---------------+-------------------+------------------+--------------------------------------+---------------------------+--------------------------+---------+-------+---------+-------------+--------------+----------+----------+
> 2 rows in set (0.04 sec)
> 2013-07-05 11:28:21,720 DEBUG [cloud.storage.VolumeManagerImpl]
> (secstorage-1:null) Checking if we need to prepare 1 volumes for
> VM[SecondaryStorageVm|s-1214-VM]
> 2013-07-05 11:28:22,001 DEBUG [storage.motion.AncientDataMotionStrategy]
> (secstorage-1:null) copyAsync inspecting src type TEMPLATE copyAsync
> inspecting dest type TEMPLATE
> 2013-07-05 11:28:22,015 DEBUG [storage.motion.AncientDataMotionStrategy]
> (secstorage-1:null) needCacheStorage true, dest at null dest role
> Primarytemplate/tmpl/1/1/routing-1/systemvmtemplate-2013-06-23-master-xen.vhd.bz2
> src role Image
> 2013-07-05 11:28:22,021 DEBUG [cache.allocator.StorageCacheRandomAllocator]
> (secstorage-1:null) Can't find cache storage in zone: 1
> 2013-07-05 11:28:22,022 DEBUG [storage.motion.AncientDataMotionStrategy]
> (secstorage-1:null) copy object failed:
> com.cloud.utils.exception.CloudRuntimeException: No cache DataStore in scope
> id 1 type ZONE
> 2013-07-05 11:28:22,025 DEBUG [storage.motion.AncientDataMotionStrategy]
> (secstorage-1:null) copy failed
> com.cloud.utils.exception.CloudRuntimeException:
> com.cloud.utils.exception.CloudRuntimeException: No cache DataStore in scope
> id 1 type ZONE
> at
> org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyObject(AncientDataMotionStrategy.java:204)
> at
> org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:351)
> at
> org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
> at
> org.apache.cloudstack.storage.volume.VolumeServiceImpl.createBaseImageAsync(VolumeServiceImpl.java:390)
> at
> org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:490)
> at
> com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2485)
> at
> com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2542)
> at
> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:852)
> at
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
> at
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:550)
> at
> com.cloud.storage.secondary.SecondaryStorageManagerImpl.startSecStorageVm(SecondaryStorageManagerImpl.java:260)
> at
> com.cloud.storage.secondary.SecondaryStorageManagerImpl.allocCapacity(SecondaryStorageManagerImpl.java:683)
> at
> com.cloud.storage.secondary.SecondaryStorageManagerImpl.expandPool(SecondaryStorageManagerImpl.java:1290)
> at
> com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:123)
> at
> com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:50)
> at
> com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:104)
> at
> com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33)
> at
> com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81)
> at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-07-05 11:28:22,076 DEBUG [cloud.storage.VolumeManagerImpl]
> (secstorage-1:null) Unable to create
> Vol[1214|vm=1214|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
> com.cloud.utils.exception.CloudRuntimeException: No cache DataStore in scope
> id 1 type ZONE
> 2013-07-05 11:28:22,076 INFO [cloud.vm.VirtualMachineManagerImpl]
> (secstorage-1:null) Unable to contact resource.
> com.cloud.exception.StorageUnavailableException: Resource [StoragePool:2] is
> unreachable: Unable to create
> Vol[1214|vm=1214|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
> com.cloud.utils.exception.CloudRuntimeException: No cache DataStore in scope
> id 1 type ZONE
> at
> com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2493)
> at
> com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2542)
> at
> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:852)
> at
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
> at
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:550)
> at
> com.cloud.storage.secondary.SecondaryStorageManagerImpl.startSecStorageVm(SecondaryStorageManagerImpl.java:260)
> at
> com.cloud.storage.secondary.SecondaryStorageManagerImpl.allocCapacity(SecondaryStorageManagerImpl.java:683)
> at
> com.cloud.storage.secondary.SecondaryStorageManagerImpl.expandPool(SecondaryStorageManagerImpl.java:1290)
> at
> com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:123)
> at
> com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:50)
> at
> com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:104)
> at
> com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33)
> at
> com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81)
> at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
