Re: Snapshots are not working after upgrading to 4.15.0

2021-06-16 Thread Suresh Anaparti
Hi Andrei,

Can you check if the storage garbage collector is enabled or not in your env 
(specified using the global setting 'storage.cleanup.enabled'). If it is 
enabled, check the interval & delay setting: 'storage.cleanup.interval' and 
'storage.cleanup.delay', and see the logs to confirm cleanup is performed or 
not.

Also, check the snapshot status / state in snapshots & snapshot_store_ref 
tables for the snapshots that are not deleted during the cleanup. Is 'removed' 
timestamp set for them in snapshots table?
 
Regards,
Suresh

On 16/06/21, 9:46 PM, "Andrei Mikhailovsky"  wrote:

Hello,

I've done some more investigation and indeed, the snapshots were not taken 
because the secondary storage was over 90% used. I have started cleaning some 
of the older volumes and noticed another problem. After removing snapshots, 
they do not seem to be removed from the secondary storage. I've removed all 
snapshots over 24 hours ago and it looks like  the disk space hasn't been freed 
up at all.

Looks like there are issues with snapshotting function after all.

Andrei




 

- Original Message -
> From: "Harikrishna Patnala" 
> To: "users" 
> Sent: Tuesday, 8 June, 2021 03:33:57
> Subject: Re: Snapshots are not working after upgrading to 4.15.0

> Hi Andrei,
> 
> Can you check the following things and let us know?
> 
> 
>  1.  Can you try creating a new volume and then create snapshot of that, 
to check
>  if this an issue with old entries
>  2.  For the snapshots which are failing can you check if you are seeing 
any
>  error messages like this "Can't find an image storage in zone with less 
than".
>  This is to check if secondary storage free space check failed.
>  3.  For the snapshots which are failing and if it is delta snapshot can 
you
>  check if its parent's snapshot entry exists in "snapshot_store_ref" 
table with
>  'parent_snapshot_id' of the current snapshot with 'store_role' "Image". 
This is
>  to find the secondary storage where the parent snapshot backup is 
located.
> 
> Regards,
> Harikrishna
> 
> From: Andrei Mikhailovsky 
> Sent: Monday, June 7, 2021 7:00 PM
> To: users 
> Subject: Snapshots are not working after upgrading to 4.15.0
> 
> Hello everyone,
> 
> I am having an issue with volume snapshots since I've upgraded to 4.15.0. 
None
> of the volumes are being snapshotted regardless if the snapshot is 
initiated
> manually or from the schedule. The strange thing is that if I manually 
take the
> snapshot, the GUI shows Success status, but the Storage>Snapshots show an 
Error
> status. Here is what I see in the management server logs:
> 
> 2021-06-07 13:55:20,022 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-81:ctx-08dd4222 job-86141/job-86143) (logid:be34ce01) 
Done
> executing com.cloud.vm.VmWorkTakeVolumeSnapshot for job-86143
> 2021-06-07 13:55:20,024 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (Work-Job-Executor-81:ctx-08dd4222 job-86141/job-86143) (logid:be34ce01) 
Remove
> job-86143 from job monitoring
> 2021-06-07 13:55:20,094 DEBUG [o.a.c.s.s.SnapshotServiceImpl]
> (BackupSnapshotTask-3:ctx-744796da) (logid:607dbb0e) Failed to copy 
snapshot
> com.cloud.utils.exception.CloudRuntimeException: can not find an image 
stores
> at
> 
org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:271)
> at
> 
org.apache.cloudstack.storage.snapshot.DefaultSnapshotStrategy.backupSnapshot(DefaultSnapshotStrategy.java:171)
> at
> 
com.cloud.storage.snapshot.SnapshotManagerImpl$BackupSnapshotTask.runInContext(SnapshotManagerImpl.java:1238)
> at
> 
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
> at
> 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
> at
> 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
> at
> 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
> at
> 
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
> at
> 
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> 
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> at
> 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at 

Re: Unable to add template to new deployment

2021-06-16 Thread Joshua Schaeffer
So Suresh's advise has pushed me in the right direction. The VM was up but the 
agent state was down. I was able to connect to the VM in order to continue 
investigating and the VM is having network issues connecting to both my load 
balancer and my secondary storage server. I don't think I'm understanding how 
the public network portion is supposed to work in my zone and could use some 
clarification. First let me explain my network setup. On my compute nodes, 
ideally, I want to use 3 NIC's:

1. A management NIC for management traffic. I was using cloudbr0 for this. 
cloudbr0 is a bridge I created that is connected to an access port on my 
switch. No vlan tagging is required to use this network (it uses VLAN 20)
2. A cloud NIC for both public and guest traffic. I was using cloudbr1 for 
this. cloudbr1 is a bridge I created that is connected to a trunk port on my 
switch. Public traffic uses VLAN 48 and guest traffic should use VLANs 400 - 
656. As the port is trunked I have to use vlan tagging for any traffic over 
this NIC.
3. A storage NIC for storage traffic. I use a bond called "bond-storage" for 
this. bond-storage is connected to an access port on my switch. No vlan tagging 
is required to use this network (it uses VLAN 96)

For now I've removed the storage NIC from the configuration to simplify my 
troubleshooting, so I should only be working with cloudbr0 and cloudbr1. To me 
the public network is a *non-RFC 1918* address that should be assigned to 
tenant VM's for external internet access. Why do system VM's need/get a public 
IP address? Can't they access all the internal CloudStack servers using the 
pod's management network?

So the first problem I'm seeing is whenever I tell CloudStack to tag VLAN 48 
for public traffic it uses the underlying bond under cloudbr1 and not the 
bridge. I don't know where it is even getting this name as I never provided it 
to CloudStack

Here is how I have it configured: 
https://drive.google.com/file/d/10PxLdp6e46_GW7oPFJwB3sQQxnvwUhvH/view?usp=sharing

Here is the message in the management logs:

2021-06-16 16:00:40,454 INFO  [c.c.v.VirtualMachineManagerImpl] 
(Work-Job-Executor-13:ctx-0f39d8e2 job-4/job-68 ctx-a4f832c5) (logid:eb82035c) 
Unable to start VM on Host[-2-Routing] due to Failed to create vnet 48: Error: 
argument "bond-services.48" is wrong: "name" not a valid ifnameCannot find 
device "bond-services.48"Failed to create vlan 48 on pif: bond-services.

This ultimately results in an error and the system VM never even starts.

If I remove the vlan tag from the configuration 
(https://drive.google.com/file/d/11tF6YIHm9xDZvQkvphi1xvHCX_X9rDz1/view?usp=sharing)
 then the VM starts and gets a public IP but without a tagged NIC it can't 
actually connect to the network. This is from inside the system VM:

root@s-9-VM:~# ip --brief addr
lo   UNKNOWN    127.0.0.1/8
eth0 UP 169.254.91.216/16
eth1 UP 10.2.21.72/22
eth2 UP 192.41.41.162/25
eth3 UP 10.2.99.15/22
root@s-9-VM:~# ping 192.41.41.129
PING 192.41.41.129 (192.41.41.129): 56 data bytes
92 bytes from s-9-VM (192.41.41.162): Destination Host Unreachable
92 bytes from s-9-VM (192.41.41.162): Destination Host Unreachable
92 bytes from s-9-VM (192.41.41.162): Destination Host Unreachable
92 bytes from s-9-VM (192.41.41.162): Destination Host Unreachable
^C--- 192.41.41.129 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss

Obviously if the network isn't functioning then it can't connect to my storage 
server and the agent never starts. How do I setup my public network so that it 
tags the packets going over cloudbr1? Also, can I not have a public IP address 
for system VM's or is this required?

I have some other issues as well like the fact that it is creating a storage 
NIC on the system VM's even though I deleted my storage network from the zone, 
but I can tackle one problem at a time. If anyone is curious or it helps 
visualize my network, here is is a little ASCII diagram of how I have the 
compute node's networking setup. Hopefully it comes across the mailing list 
correctly and not all mangled:

+===
|
|    enp3s0f0 (eth) enp3s0f1 (eth) enp65s0f0 (eth)    enp65s0f1 (eth)   
 enp71s0 (eth)    enp72s0 (eth)
|   |  |   |  | 
   |  |
|   |  |   ++-+ 
   ++-+
|   |  |    |   
    |
|   |  |   bond-services (bond) 
    |
|   |  |    |   
    |
|   |  |   

[VOTE] Apache CloudStack 4.15.1.0 (RC2)

2021-06-16 Thread Rohit Yadav
Hi All,

I've created a 4.15.1.0 release, with the following artifacts up for a vote:

Git Branch:
https://github.com/apache/cloudstack/tree/4.15.1.0-RC20210616T2128
Commit SHA:
3afd37022b9dac52cd146dccada6012e47a80232

Source release (checksums and signatures are available at the same
location):
https://dist.apache.org/repos/dist/dev/cloudstack/4.15.1.0/

PGP release keys (signed using 5ED1E1122DC5E8A4A45112C2484248210EE3D884):
https://dist.apache.org/repos/dist/release/cloudstack/KEYS

The vote will be open for the next week until 22 June 2021.

For sanity in tallying the vote, can PMC members please be sure to indicate
"(binding)" with their vote?

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)

For users convenience, the packages from this release candidate and 4.15.1
systemvmtemplates are available here:
https://download.cloudstack.org/testing/4.15.1.0-RC2/
https://download.cloudstack.org/systemvm/4.15/

Documentation is not published yet, but the following may be referenced for
upgrade related tests: (there's a new 4.15.1 systemvmtemplate to be
registered prior to upgrade)
https://github.com/apache/cloudstack-documentation/tree/4.15/source/upgrading/upgrade

Regards.


Re: Snapshots are not working after upgrading to 4.15.0

2021-06-16 Thread Andrei Mikhailovsky
Hello,

I've done some more investigation and indeed, the snapshots were not taken 
because the secondary storage was over 90% used. I have started cleaning some 
of the older volumes and noticed another problem. After removing snapshots, 
they do not seem to be removed from the secondary storage. I've removed all 
snapshots over 24 hours ago and it looks like  the disk space hasn't been freed 
up at all.

Looks like there are issues with snapshotting function after all.

Andrei



- Original Message -
> From: "Harikrishna Patnala" 
> To: "users" 
> Sent: Tuesday, 8 June, 2021 03:33:57
> Subject: Re: Snapshots are not working after upgrading to 4.15.0

> Hi Andrei,
> 
> Can you check the following things and let us know?
> 
> 
>  1.  Can you try creating a new volume and then create snapshot of that, to 
> check
>  if this an issue with old entries
>  2.  For the snapshots which are failing can you check if you are seeing any
>  error messages like this "Can't find an image storage in zone with less 
> than".
>  This is to check if secondary storage free space check failed.
>  3.  For the snapshots which are failing and if it is delta snapshot can you
>  check if its parent's snapshot entry exists in "snapshot_store_ref" table 
> with
>  'parent_snapshot_id' of the current snapshot with 'store_role' "Image". This 
> is
>  to find the secondary storage where the parent snapshot backup is located.
> 
> Regards,
> Harikrishna
> 
> From: Andrei Mikhailovsky 
> Sent: Monday, June 7, 2021 7:00 PM
> To: users 
> Subject: Snapshots are not working after upgrading to 4.15.0
> 
> Hello everyone,
> 
> I am having an issue with volume snapshots since I've upgraded to 4.15.0. None
> of the volumes are being snapshotted regardless if the snapshot is initiated
> manually or from the schedule. The strange thing is that if I manually take 
> the
> snapshot, the GUI shows Success status, but the Storage>Snapshots show an 
> Error
> status. Here is what I see in the management server logs:
> 
> 2021-06-07 13:55:20,022 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Work-Job-Executor-81:ctx-08dd4222 job-86141/job-86143) (logid:be34ce01) Done
> executing com.cloud.vm.VmWorkTakeVolumeSnapshot for job-86143
> 2021-06-07 13:55:20,024 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (Work-Job-Executor-81:ctx-08dd4222 job-86141/job-86143) (logid:be34ce01) 
> Remove
> job-86143 from job monitoring
> 2021-06-07 13:55:20,094 DEBUG [o.a.c.s.s.SnapshotServiceImpl]
> (BackupSnapshotTask-3:ctx-744796da) (logid:607dbb0e) Failed to copy snapshot
> com.cloud.utils.exception.CloudRuntimeException: can not find an image stores
> at
> org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:271)
> at
> org.apache.cloudstack.storage.snapshot.DefaultSnapshotStrategy.backupSnapshot(DefaultSnapshotStrategy.java:171)
> at
> com.cloud.storage.snapshot.SnapshotManagerImpl$BackupSnapshotTask.runInContext(SnapshotManagerImpl.java:1238)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> 2021-06-07 13:55:20,152 DEBUG [c.c.s.s.SnapshotManagerImpl]
> (BackupSnapshotTask-3:ctx-744796da) (logid:607dbb0e) Backing up of snapshot
> failed, for snapshot with ID 53531, left with 2 more attempts
> 
> 
> I've checked and the Secondary storage is configured and visible in the GUI. I
> can also mount it manually from the management server and a couple of host
> servers that I've tested. In addition, I can successfully upload an ISO image
> and that registers just fine and I can create new VMs using the newly uploaded
> ISO image.
> 
> I've had no such problems with 4.13.x ACS, so the issue seems to have been
> introduced after doing the upgrade to 4.15.0.
> 
> Could you please let me know how do I fix the issue?
> 
> Cheers
> 
> andrei


Re: Unable to add template to new deployment

2021-06-16 Thread Andrija Panic
" There is no secondary storage VM for downloading template to image store
LXC_SEC_STOR1 "

So next step to investigate why there is no SSVM (can hosts access the
secondary storage NFS, can they access the Primary Storage, etc - those
tests you can do manually) - and as Suresh advised - one it's up, is it all
green (COnnected / Up state).

Best,


instance backup designs?

2021-06-16 Thread Yordan Kostov
Hey everyone,

I was wondering what choice does one have for backup when 
underlying hypervisor is XenServer/XCP-NG?
Any high level ideas or just sharing any doc that may exist 
will be great!

Best regards,
Jordan


RE: [DISCUSS] Moving to OpenVPN as the remote access VPN provider

2021-06-16 Thread Sean Lair
I would love to see OpenVPN as the client VPN.  We consider the current Client 
VPN unusable.  We use OpenVPN with OPNsense firewalls and it has been 
rock-solid.


-Original Message-
From: Rohit Yadav  
Sent: Friday, June 11, 2021 12:40 PM
To: users@cloudstack.apache.org; d...@cloudstack.apache.org
Subject: [DKIM Fail] Re: [DISCUSS] Moving to OpenVPN as the remote access VPN 
provider

Hi PL,

You can check the ikev2 support in 4.15+ here: 
https://github.com/apache/cloudstack/pull/4953

I think a generic VPN framework-provider feature is probably what we need (i.e. 
to let user or admin decide what VPN provider they want, supporting 
strongswan/ipsec and openvpn) so I'm not trying to defend OpenVPN here but your 
comments on OpenVPN are incorrect. It is widely used (in many projects incl. 
pfSense) and both server/clients are opensource and not proprietary afaik (GPL 
or AGPL license, I'm not sure about platform-specific clients (the GUI ones) 
but I checked the CLI clients are opensource):
https://github.com/OpenVPN/openvpn
https://github.com/OpenVPN/openvpn3

One key requirement for whatever VPN provider we support is that it should be 
free and opensource and available on Debian (for use in the systemvmtemplate) 
and OpenVPN fits that requirement. The package is available on Debian: 
https://packages.debian.org/buster-backports/openvpn

Regards.


From: Pierre-Luc Dion 
Sent: Friday, June 11, 2021 20:10
To: users@cloudstack.apache.org 
Cc: dev 
Subject: Re: [DISCUSS] Moving to OpenVPN as the remote access VPN provider

Just to be sure, what CloudStack > v4.15 uses Strongswan/l2tp or
strongswan/ikev2 ?

Because l2tp became complicated to configure on native vpn clients on some 
OSes, kind of deprecated remote management VPN, compared to IKEv2.
I'm a bit concerned about OpenVPN for the clients, what if binaries become 
subscription based availability or become proprietary ?

For sure we need the option to select what type of VPN solution to offer when 
deploying a cloud.

>From my perspective I cannot use/offer OpenVPN as a solution to my customers 
>because it involves forcing them to download third party software on their 
>workstations and I don't want to be responsible for a security breach on their 
>workstation because of a requirement for 3rd party software that we don't 
>control.



On Fri, Jun 11, 2021 at 10:14 AM Rohit Yadav 
wrote:

> Thanks all for the feedback so far, looks like the majority of people 
> on the thread would prefer OpenVPN but for s2s they may continue to 
> prefer strongswan/ipsec for site-to-site VPC feature. If we're unable 
> to reach consensus then a general-purpose provider-framework may be 
> more flexible to the end-user or admin (to select which VPN provider 
> they want for their network, we heard in this thread - openvpn, 
> strongswan/l2tp, wireguard, and maybe other providers in future).
>
> Btw, ikev2 is supported now with strongswan with this -
> https://github.com/apache/cloudstack/pull/4953
>
> My personal opinion: As user of most of these VPN providers, I 
> personally like OpenVPN which I found to be easier to use both on 
> desktop/laptop and on phone. With openvpn as the default I imagine in 
> CloudStack I could enable VPN for a network and CloudStack gives me an 
> option to download a .ovpn file which I can import in my openvpn 
> client (desktop, phone, cli...) click connect to connect to the VPN. 
> For certificate generation/storage, the CA framework could be used so 
> the openvpn server certs are the same across network restarts (with 
> cleanup). I think a process like this could be simpler than what we've 
> right now, and the ovpn download+import workflow would be easier than 
> what we'll get from either strongswan/current or wireguard. While I 
> like the simplicity of wireguard, which is more like SSH setup I 
> wouldn't mind doing setup on individual VMs (much like setting up ssh key) or 
> use something like TailScale.
>
>
> Regards.
>
> 
> From: Gabriel Bräscher 
> Sent: Friday, June 11, 2021 19:28
> To: dev 
> Cc: users 
> Subject: Re: [DISCUSS] Moving to OpenVPN as the remote access VPN 
> provider
>
> I understand that OpenVPN is a great option and far adopted.
> I am  ++1 in allowing Users/Admins to choose which VPN provider suits 
> them best; creating an offering (or global settings) that would allow 
> setting which VPN provider will be used would be awesome.
>
> I understand that OpenVPN is a great option and far adopted; however, 
> I would be -1 if this would impact on removing support for Strongswan 
> -- which from what I understood is not the proposal, but saying anyway 
> to be sure.
>
> Thanks for raising this proposal/discussion, Rohit.
>
> Cheers,
> Gabriel.
>
>
> Em sex., 11 de jun. de 2021 às 08:46, Pierre-Luc Dion 
>  >
> escreveu:
>
> > Hello,
> >
> > Daan, I agree we should provide capability to select the vpn 
> > solution to use, the 

Re: Unable to add template to new deployment

2021-06-16 Thread Suresh Anaparti
HI Joshua,

What is the agent state of secondarystoragevm in the UI. If it is not 'Up', you 
can SSH to it and check the status of agent service there (service cloud 
status). Have you noticed any errors in the ssvm log at ' 
/var/log/cloud/cloud.out' if the service is running.


Regards,
Suresh

On 16/06/21, 4:11 AM, "Joshua Schaeffer"  wrote:

I've setup ACS 4.15 and created the first zone on it using the wizard 
through primate. The zone is enabled and isn't showing any issues in the UI. I 
can see two system vm's running (secondarystoragevm and consoleproxy) and can 
SSH into the VM's from the compute node. However when I try to add a template 
through the UI I get the following error:

There is no secondary storage VM for downloading template to image store 
LXC_SEC_STOR1

And on the controller I can see the corresponding log entries when I try to 
submit the new template:

2021-06-15 22:13:35,884 DEBUG [c.c.a.ApiServlet] 
(qtp1644231115-348:ctx-aab81632) (logid:ac328268) ===START===  172.16.44.18 -- 
GET  
name=Ubuntu+20.04=Ubuntu+20.04+(Focal)+64-bit=9f6f5b49-0e12-4af4-a13b-2ace6c47de43=LXC=TAR=aa18f3ad-cd73-11eb-b1da-5254008f72d5=true=getUploadParamsForTemplate=json
2021-06-15 22:13:35,924 DEBUG [c.c.a.ApiServer] 
(qtp1644231115-348:ctx-aab81632 ctx-da7281d3) (logid:ac328268) CIDRs from which 
account 'Acct[f8d6949d-cd74-11eb-b1da-5254008f72d5-admin]' is allowed to 
perform API calls: 0.0.0.0/0,::/0
2021-06-15 22:13:36,093 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl] 
(qtp1644231115-348:ctx-aab81632 ctx-da7281d3) (logid:ac328268) template 203 is 
not in store:1, type:Image
2021-06-15 22:13:36,138 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl] 
(qtp1644231115-348:ctx-aab81632 ctx-da7281d3) (logid:ac328268) template 203 is 
already in store:1, type:Image
2021-06-15 22:13:36,142 WARN  [c.c.t.HypervisorTemplateAdapter] 
(qtp1644231115-348:ctx-aab81632 ctx-da7281d3) (logid:ac328268) There is no 
secondary storage VM for downloading template to image store LXC_SEC_STOR1
2021-06-15 22:13:36,146 DEBUG [c.c.u.d.T.Transaction] 
(qtp1644231115-348:ctx-aab81632 ctx-da7281d3) (logid:ac328268) Rolling back the 
transaction: Time = 98 Name =  qtp1644231115-348; called by 
-TransactionLegacy.rollback:888-TransactionLegacy.removeUpTo:831-TransactionLegacy.close:655-Transaction.execute:38-Transaction.execute:47-HypervisorTemplateAdapter.createTemplateForPostUpload:298-TemplateManagerImpl.registerPostUploadInternal:361-TemplateManagerImpl.registerTemplateForPostUpload:423-NativeMethodAccessorImpl.invoke0:-2-NativeMethodAccessorImpl.invoke:62-DelegatingMethodAccessorImpl.invoke:43-Method.invoke:566
2021-06-15 22:13:36,229 ERROR [c.c.a.ApiServer] 
(qtp1644231115-348:ctx-aab81632 ctx-da7281d3) (logid:ac328268) unhandled 
exception executing api command: [Ljava.lang.String;@2ad13bf5
com.cloud.utils.exception.CloudRuntimeException: There is no secondary 
storage VM for downloading template to image store LXC_SEC_STOR1
at 
com.cloud.template.HypervisorTemplateAdapter$1.doInTransaction(HypervisorTemplateAdapter.java:363)
at 
com.cloud.template.HypervisorTemplateAdapter$1.doInTransaction(HypervisorTemplateAdapter.java:298)
at com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:50)
at com.cloud.utils.db.Transaction.execute(Transaction.java:40)
at com.cloud.utils.db.Transaction.execute(Transaction.java:47)
at 
com.cloud.template.HypervisorTemplateAdapter.createTemplateForPostUpload(HypervisorTemplateAdapter.java:298)
at 
com.cloud.template.TemplateManagerImpl.registerPostUploadInternal(TemplateManagerImpl.java:361)
at 
com.cloud.template.TemplateManagerImpl.registerTemplateForPostUpload(TemplateManagerImpl.java:423)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at 
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at 
org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at 
com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at 

Re: Failure creating a template from a snapshot

2021-06-16 Thread Suresh Anaparti
Hi Jeremy,

Can you share the env details (mainly: cloudstack version, primary storage, 
hypervisor) where this error is seen. The snapshot db entries and few more log 
snippets before the error would be helpful, to debug the issue.

Regards,
Suresh

On 16/06/21, 9:46 AM, "Jeremy Hansen"  wrote:


I see this error:

Create template
(GSA Security Scanner) Failed to copy snapshot:java.lang.RuntimeException: 
InvocationTargetException when invoking RPC callback for command: 
copySnapshotAsyncCallback

Any clues on this?

Thanks
-jeremy