[
https://issues.apache.org/jira/browse/CLOUDSTACK-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sanjeev N updated CLOUDSTACK-4400:
----------------------------------
Attachment: management-server.rar
cloud.dmp
> [object_store_refactor] Three entries for one template in template_store_ref
> when MS was restarting during template download
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-4400
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4400
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Storage Controller, Template
> Affects Versions: 4.2.0
> Environment: Latest build from ACS 4.2 branch
> Reporter: Sanjeev N
> Priority: Critical
> Fix For: 4.2.0
>
> Attachments: cloud.dmp, management-server.rar
>
>
> Three entries for one template in template_store_ref when MS was restarting
> during template download.
> Restarting management server when the template download was in progress,
> created three entries in template_store_ref table for the same template and
> in UI tamplate Ready is set to No even though it is downloaded to S3
> Steps to Reproduce:
> ================
> 1.Bring up CS with two or three zones
> 2.Register template any one of the zones
> 3.Restart the management server when the template download was in progres
> Result:
> =====
> After restart template sync initiated template download via all the SSVMs and
> created multiple intries in the template_store_ref for the same template.
> In UI template is not showing as ready even though it is download to S3 via
> one of the ssvms.
> Observations:
> =============
> After management server restart, it found template 208 is not downlaoded
> 2013-08-19 08:54:01,695 INFO [storage.image.TemplateServiceImpl]
> (AgentConnectTaskPool-3:null) Template Sync did not find
> 208-2-0fe7ea1d-5e21-33cd-b328-e4510821608d on image store 3, may request
> download based on available hypervisor types
> 2013-08-19 08:54:01,709 INFO [storage.image.TemplateServiceImpl]
> (AgentConnectTaskPool-3:null) Removing leftover template
> 208-2-0fe7ea1d-5e21-33cd-b328-e4510821608d entry from template store table
> 2013-08-19 08:54:01,722 INFO [storage.image.TemplateServiceImpl]
> (AgentConnectTaskPool-6:null) Template Sync did not find
> 208-2-0fe7ea1d-5e21-33cd-b328-e4510821608d on image store 3, may request
> download based on available hypervisor types
> 2013-08-19 08:54:01,722 INFO [storage.image.TemplateServiceImpl]
> (AgentConnectTaskPool-6:null) Removing leftover template
> 208-2-0fe7ea1d-5e21-33cd-b328-e4510821608d entry from template store table
> 2013-08-19 08:54:01,735 INFO [storage.image.TemplateServiceImpl]
> (AgentConnectTaskPool-4:null) Template Sync did not find
> 208-2-0fe7ea1d-5e21-33cd-b328-e4510821608d on image store 3, may request
> download based on available hypervisor types
> 2013-08-19 08:54:01,735 INFO [storage.image.TemplateServiceImpl]
> (AgentConnectTaskPool-4:null) Removing leftover template
> 208-2-0fe7ea1d-5e21-33cd-b328-e4510821608d entry from template store table
> 2013-08-19 08:54:01,747 INFO [storage.image.TemplateServiceImpl]
> (AgentConnectTaskPool-6:null) Downloading template
> 208-2-0fe7ea1d-5e21-33cd-b328-e4510821608d to image store imagestore
> 2013-08-19 08:54:01,751 INFO [storage.image.TemplateServiceImpl]
> (AgentConnectTaskPool-4:null) Downloading template
> 208-2-0fe7ea1d-5e21-33cd-b328-e4510821608d to image store imagestore
> 2013-08-19 08:54:01,753 INFO [storage.image.TemplateServiceImpl]
> (AgentConnectTaskPool-3:null) Downloading template
> 208-2-0fe7ea1d-5e21-33cd-b328-e4510821608d to image store imagestore
> 2013-08-19 08:54:02,068 DEBUG [storage.image.TemplateDataFactoryImpl]
> (AgentConnectTaskPool-4:null) template 208 is already in store:3, type:Image
> 2013-08-19 08:54:02,080 DEBUG [storage.image.TemplateDataFactoryImpl]
> (AgentConnectTaskPool-3:null) template 208 is already in store:3, type:Image
> 2013-08-19 08:54:02,069 DEBUG [storage.image.TemplateDataFactoryImpl]
> (AgentConnectTaskPool-6:null) template 208 is already in store:3, type:Image
> 2013-08-19 08:54:02,163 DEBUG [storage.image.BaseImageStoreDriverImpl]
> (AgentConnectTaskPool-6:null) Downloading template to data store 3
> 2013-08-19 08:54:02,164 DEBUG [storage.image.BaseImageStoreDriverImpl]
> (AgentConnectTaskPool-4:null) Downloading template to data store 3
> 2013-08-19 08:54:02,219 DEBUG [agent.transport.Request]
> (AgentConnectTaskPool-3:null) Seq 3-1325465607: Sending { Cmd , MgmtId:
> 6615759585382, via: 3, Ver: v1, Flags: 100111,
> [{"com.cloud.agent.api.ReadyCommand":{"dcId":1,"hostId":3,"wait":0}}] }
> 2013-08-19 08:54:02,233 DEBUG [agent.transport.Request]
> (AgentManager-Handler-3:null) Seq 3-1325465607: Processing: { Ans: , MgmtId:
> 6615759585382, via: 3, Ver: v1, Flags: 110,
> [{"com.cloud.agent.api.ReadyAnswer":{"result":true,"wait":0}}] }
> 2013-08-19 08:54:02,233 DEBUG [agent.manager.AgentAttache]
> (AgentManager-Handler-3:null) Seq 3-1325465607: No more commands found
> 2013-08-19 08:54:02,233 DEBUG [agent.transport.Request]
> (AgentConnectTaskPool-3:null) Seq 3-1325465607: Received: { Ans: , MgmtId:
> 6615759585382, via: 3, Ver: v1, Flags: 110, { ReadyAnswer } }
> 2013-08-19 08:54:02,233 DEBUG [cloud.host.Status]
> (AgentConnectTaskPool-3:null) Transition:[Resource state = Enabled, Agent
> event = Ready, Host id = 3, name = s-1-VM]
> 2013-08-19 08:54:02,277 DEBUG [cloud.host.Status]
> (AgentConnectTaskPool-3:null) Agent status update: [id = 3; name = s-1-VM;
> old status = Connecting; event = Ready; new status = Up; old update count =
> 11; new update count = 12]
> 2013-08-19 08:54:02,347 DEBUG [agent.transport.Request]
> (AgentConnectTaskPool-3:null) Seq 3-1325465608: Sending { Cmd , MgmtId:
> 6615759585382, via: 3, Ver: v1, Flags: 100111,
> [{"com.cloud.agent.api.ReadyCommand":{"dcId":1,"hostId":3,"wait":0}}] }
> 2013-08-19 08:54:02,374 DEBUG [storage.image.BaseImageStoreDriverImpl]
> (AgentConnectTaskPool-6:null) Performing image store createTemplate async
> callback
> 2013-08-19 08:54:02,380 DEBUG [storage.image.BaseImageStoreDriverImpl]
> (AgentConnectTaskPool-4:null) Performing image store createTemplate async
> callback
> 2013-08-19 08:54:02,390 DEBUG [agent.transport.Request]
> (AgentManager-Handler-5:null) Seq 3-1325465608: Processing: { Ans: , MgmtId:
> 6615759585382, via: 3, Ver: v1, Flags: 110,
> [{"com.cloud.agent.api.ReadyAnswer":{"result":true,"wait":0}}] }
> 2013-08-19 08:54:02,391 DEBUG [agent.manager.AgentAttache]
> (AgentManager-Handler-5:null) Seq 3-1325465608: No more commands found
> 2013-08-19 08:54:02,391 DEBUG [agent.transport.Request]
> (AgentConnectTaskPool-3:null) Seq 3-1325465608: Received: { Ans: , MgmtId:
> 6615759585382, via: 3, Ver: v1, Flags: 110, { ReadyAnswer } }
> 2013-08-19 08:54:02,423 DEBUG [cloudstack.storage.RemoteHostEndPoint]
> (AgentConnectTaskPool-6:null) Sending command
> org.apache.cloudstack.storage.command.DownloadCommand to host: 9
> 2013-08-19 08:54:02,432 DEBUG [cloudstack.storage.RemoteHostEndPoint]
> (AgentConnectTaskPool-4:null) Sending command
> org.apache.cloudstack.storage.command.DownloadCommand to host: 3
> 2013-08-19 08:54:02,465 DEBUG [agent.transport.Request]
> (AgentConnectTaskPool-6:null) Seq 9-275972102: Sending { Cmd , MgmtId:
> 6615759585382, via: 9, Ver: v1, Flags: 100011,
> [{"org.apache.cloudstack.storage.command.DownloadCommand":{"hvm":true,"description":"win8","maxDownloadSizeInBytes":53687091200,"id":208,"resourceType":"TEMPLATE","installPath":"template/tmpl/2/208/208-2-0fe7ea1d-5e21-33cd-b328-e4510821608d","_store":{"com.cloud.agent.api.to.S3TO":{"id":3,"uuid":"3c9432fe-ff7c-42d0-b886-5006386950ae","endPoint":"10.147.29.56:8080","bucketName":"imagestore","httpsFlag":false,"created":"Aug
> 16, 2013 7:53:59
> AM","enableRRS":false}},"url":"http://10.147.28.7/templates/Windows8/Windows864.vhd","format":"VHD","accountId":2,"name":"208-2-0fe7ea1d-5e21-33cd-b328-e4510821608d","wait":0}}]
> }
> 2013-08-19 08:54:02,476 DEBUG [agent.transport.Request]
> (AgentConnectTaskPool-4:null) Seq 3-1325465609: Sending { Cmd , MgmtId:
> 6615759585382, via: 3, Ver: v1, Flags: 100011,
> [{"org.apache.cloudstack.storage.command.DownloadCommand":{"hvm":true,"description":"win8","maxDownloadSizeInBytes":53687091200,"id":208,"resourceType":"TEMPLATE","installPath":"template/tmpl/2/208/208-2-0fe7ea1d-5e21-33cd-b328-e4510821608d","_store":{"com.cloud.agent.api.to.S3TO":{"id":3,"uuid":"3c9432fe-ff7c-42d0-b886-5006386950ae","endPoint":"10.147.29.56:8080","bucketName":"imagestore","httpsFlag":false,"created":"Aug
> 16, 2013 7:53:59
> AM","enableRRS":false}},"url":"http://10.147.28.7/templates/Windows8/Windows864.vhd","format":"VHD","accountId":2,"name":"208-2-0fe7ea1d-5e21-33cd-b328-e4510821608d","wait":0}}]
> }
> mysql> select * from template_store_ref where template_id=208;
> +----+----------+-------------+---------------------+---------------------+--------------------------------------+--------------+------------+------------+---------------+----------------------+-----------+-------------------------------------------------------------------------------+-------------------------------------------------------------------------------+------+--------------+----------------------+-----------+-----------+---------+--------------+---------+---------------------+
> | id | store_id | template_id | created | last_updated |
> job_id | download_pct | size | store_role
> | physical_size | download_state | error_str | local_path
> | install_path
> | url | download_url
> | download_url_created | state | destroyed | is_copy | update_count |
> ref_cnt | updated |
> +----+----------+-------------+---------------------+---------------------+--------------------------------------+--------------+------------+------------+---------------+----------------------+-----------+-------------------------------------------------------------------------------+-------------------------------------------------------------------------------+------+--------------+----------------------+-----------+-----------+---------+--------------+---------+---------------------+
> | 24 | 3 | 208 | 2013-08-19 12:54:02 | 2013-08-19 13:14:48 |
> bd84f245-2fef-4f1c-8d6b-33de36b1843d | 100 | 9436754432 | Image
> | 9436754432 | DOWNLOADED | |
> template/tmpl/2/208/208-2-0fe7ea1d-5e21-33cd-b328-e4510821608d/Windows864.vhd
> |
> template/tmpl/2/208/208-2-0fe7ea1d-5e21-33cd-b328-e4510821608d/Windows864.vhd
> | NULL | NULL | NULL | Creating | 0 |
> 0 | 1 | 0 | 2013-08-19 12:54:02 |
> | 25 | 3 | 208 | 2013-08-19 12:54:02 | 2013-08-19 13:14:18 |
> a43fb0d4-d973-4086-8c78-9b643cf492d3 | 98 | 0 | Image
> | 0 | DOWNLOAD_IN_PROGRESS | |
> template/tmpl/2/208/208-2-0fe7ea1d-5e21-33cd-b328-e4510821608d/Windows864.vhd
> | template/tmpl/2/208/208-2-0fe7ea1d-5e21-33cd-b328-e4510821608d
> | NULL | NULL | NULL | Ready | 0 |
> 0 | 3 | 0 | 2013-08-19 13:14:48 |
> | 26 | 3 | 208 | 2013-08-19 12:54:02 | 2013-08-19 13:14:38 |
> bd84f245-2fef-4f1c-8d6b-33de36b1843d | 100 | 9436754432 | Image
> | 9436754432 | DOWNLOADED | |
> template/tmpl/2/208/208-2-0fe7ea1d-5e21-33cd-b328-e4510821608d/Windows864.vhd
> |
> template/tmpl/2/208/208-2-0fe7ea1d-5e21-33cd-b328-e4510821608d/Windows864.vhd
> | NULL | NULL | NULL | Allocated | 0 |
> 0 | 0 | 0 | NULL |
> +----+----------+-------------+---------------------+---------------------+--------------------------------------+--------------+------------+------------+---------------+----------------------+-----------+-------------------------------------------------------------------------------+-------------------------------------------------------------------------------+------+--------------+----------------------+-----------+-----------+---------+--------------+---------+---------------------+
> 3 rows in set (0.00 sec)
> Attaching management server log file and cloud DB.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
