[ https://issues.apache.org/jira/browse/CLOUDSTACK-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13562024#comment-13562024 ]
Marcus Sorensen commented on CLOUDSTACK-1056: --------------------------------------------- John, this should fix it. Let me know if there are issues. https://reviews.apache.org/r/9101/ > S3 secondary storage fails to upload systemvm template due to KVMHA directory > ----------------------------------------------------------------------------- > > Key: CLOUDSTACK-1056 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-1056 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: KVM, Storage Controller > Affects Versions: 4.1.0 > Environment: KVM advanced zone > Reporter: Marcus Sorensen > Assignee: John Burwell > Fix For: 4.1.0 > > > KVM creates this KVMHA directory in the template/tmpl/1/3 directory on > secondary storage, where the system vm is prepared. It's some sort of > heartbeat cluster thingy for KVM. This seems to be causing the S3 upload to > break: > SSVM log: > 20:20:58,779 ERROR NfsSecondaryStorageResource:426 - Failed to upload > template id 3 > com.amazonaws.AmazonClientException: Unable to calculate MD5 hash: > /mnt/SecStorage/1cf3b9fa-3024-329a-991a-7bdf7b3f14f7/template/tmpl/1/3/KVMHA > (Is a directory) > at > com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1012) > at > com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:963) > at com.cloud.utils.S3Utils.putDirectory(S3Utils.java:282) > at > com.cloud.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:396) > at > com.cloud.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:210) > at com.cloud.agent.Agent.processRequest(Agent.java:525) > at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852) > at com.cloud.utils.nio.Task.run(Task.java:83) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > Caused by: java.io.FileNotFoundException: > /mnt/SecStorage/1cf3b9fa-3024-329a-991a-7bdf7b3f14f7/template/tmpl/1/3/KVMHA > (Is a directory) > Management Server log: > INFO [cloud.template.S3SyncTask] (s3-template-sync-1:) Uploading template > SystemVM Template (KVM) (id: 3) to S3. > ERROR [storage.s3.S3ManagerImpl] (s3-template-sync-1:) Failed to upload > template id 3 for zone id 1 to S3. > com.cloud.utils.exception.CloudRuntimeException: Failed to upload template id > 3 to S3 from secondary storage due to Failed to upload template id 3. > at com.cloud.storage.s3.S3ManagerImpl$3.call(S3ManagerImpl.java:622) > at com.cloud.storage.s3.S3ManagerImpl$3.call(S3ManagerImpl.java:606) > at com.cloud.utils.db.GlobalLock.executeWithLock(GlobalLock.java:225) > at > com.cloud.utils.db.GlobalLock.executeWithNoWaitLock(GlobalLock.java:240) > at > com.cloud.storage.s3.S3ManagerImpl.uploadTemplateToS3FromSecondaryStorage(S3ManagerImpl.java:605) > at com.cloud.template.S3SyncTask.run(S3SyncTask.java:80) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:722) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira