[GitHub] cloudstack issue #892: CLOUDSTACK-8910: The reserved_capacity field increase...

[cloudmonger]

Github user cloudmonger commented on the issue:

https://github.com/apache/cloudstack/pull/892
  
### ACS CI BVT Run
 **Sumarry:**
 Build Number 421
 Hypervisor xenserver
 NetworkType Advanced
 Passed=103
 Failed=2
 Skipped=7

_Link to logs Folder (search by build_no):_ 
https://www.dropbox.com/sh/yj3wnzbceo9uef2/AAB6u-Iap-xztdm6jHX9SjPja?dl=0


**Failed tests:**
* test_routers_network_ops.py

 * test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true Failing since 2 
runs

 * test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false Failing since 
3 runs


**Skipped tests:**
test_01_test_vm_volume_snapshot
test_vm_nic_adapter_vmxnet3
test_static_role_account_acls
test_11_ss_nfs_version_on_ssvm
test_nested_virtualization_vmware
test_3d_gpu_support
test_deploy_vgpu_enabled_vm

**Passed test suits:**
test_deploy_vm_with_userdata.py
test_affinity_groups_projects.py
test_portable_publicip.py
test_over_provisioning.py
test_global_settings.py
test_scale_vm.py
test_service_offerings.py
test_routers_iptables_default_policy.py
test_loadbalance.py
test_routers.py
test_reset_vm_on_reboot.py
test_deploy_vms_with_varied_deploymentplanners.py
test_network.py
test_router_dns.py
test_non_contigiousvlan.py
test_login.py
test_deploy_vm_iso.py
test_list_ids_parameter.py
test_public_ip_range.py
test_multipleips_per_nic.py
test_regions.py
test_affinity_groups.py
test_network_acl.py
test_pvlan.py
test_volumes.py
test_nic.py
test_deploy_vm_root_resize.py
test_resource_detail.py
test_secondary_storage.py
test_vm_life_cycle.py
test_disk_offerings.py


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: :[VOTE] Apache Cloudstack 4.10.0.0

[Rajani Karuturi]

Thanks for the update mike.

@wido, weizhou
Can you take a look at the issue please?

~Rajani
Sent from phone.

On 4 Mar 2017 3:25 a.m., "Tutkowski, Mike" 
wrote:

> -1 (binding)
>
> Per my previous e-mails concerning this issue, here is a ticket I created:
>
> https://issues.apache.org/jira/browse/CLOUDSTACK-9821
>
> We should get someone from the networking side of things to investigate.
>
> Thanks!
>
> On 3/3/17, 1:28 AM, "Rohit Yadav"  wrote:
>
> -1 (binding)
>
>
> All, I've found an upgrade blocker. Pre 4.6 users are required to seed
> 4.6 systemvmtemplate to proceed with the upgrade otherwise upgrade fails,
> and from 4.9 upgrade to 4.10 does no check/enforcement that 4.10 based
> systemvmtemplate has been seeded/registered, nor the minimum required
> systemvmtemplate version is changed from 4.6.0 to 4.10.0.
>
>
> After we have merged the strongswan/java8 PR, I had updated the
> upgrade docs on how to upgrade the systemvmtemplate here:
>
> http://docs.cloudstack.apache.org/projects/cloudstack-
> release-notes/en/4.10/upgrade/upgrade-4.9.html
>
>
> Using the above, I've tried to fix these issues here, please review
> and merge for RC2:
>
> https://github.com/apache/cloudstack/pull/1983
>
>
> With above fix, the
> aim is that users only seed the 4.10 systemvmtemplate before upgrade and
> post-upgrade the upgrade paths fix the entries, global setting etc.
>
>
> Regards.
>
> 
> From: Tutkowski, Mike 
> Sent: 02 March 2017 22:39:08
> To: dev@cloudstack.apache.org
> Subject: Re: :[VOTE] Apache Cloudstack 4.10.0.0
>
> I rolled back to my master branch at 
> da66b06e7d562393da2e4b52206943f8bad49d10
> and it works.
>
> It appears something that went into after that commit has broken this.
> It looks like this SHA is about two weeks old and that 43 commits have gone
> into master since it.
>
> On 3/2/17, 7:06 AM, "Tutkowski, Mike" 
> wrote:
>
> According to where the code fails, though, it appears to be a
> networking problem. If I set a breakpoint before the failure and change a
> variable to say that security groups are not being used, then the VM starts.
>
> I think this is a recently introduced problem because I have
> another branch based off of a slightly older version of master and it works
> fine here.
>
> > On Mar 2, 2017, at 6:51 AM, Pierre-Luc Dion 
> wrote:
> >
> > Hi Mike,
> > Try vm with at least 512MB for memory.
> >
> >> On Mar 1, 2017 15:01, "Tutkowski, Mike" <
> mike.tutkow...@netapp.com> wrote:
> >>
> >> I see the following exception when trying to deploy a user VM
> in a Basic
> >> Zone with two XenServer 6.5 hosts in one cluster. My system VMs
> have all
> >> deployed properly. The user template gets downloaded fine. I
> can see the
> >> user VM begin to start on a XenServer host, then it goes away.
> We then
> >> automatically try on the other host. I can see the VM begin to
> start there
> >> for a moment, then it goes away.
> >>
> >> I am just deploying the user VM’s template and root disk to NFS
> (same
> >> place where the template and root disks of my system VMs are).
> >>
> >> I am using the built-in XenServer CentOS 5.6 (64 bit) template
> with 1
> >> vCPU, 500 MHz, and 256 MB memory.
> >>
> >> WARN  [c.c.a.r.v.VirtualRoutingResource]
> (DirectAgent-7:ctx-35aded78)
> >> (logid:aab9c320) Expected 1 answers while executing
> VmDataCommand but
> >> received 2
> >> WARN  [c.c.v.VirtualMachinePowerStateSyncImpl]
> (DirectAgentCronJob-14:ctx-27fb1ac3)
> >> (logid:2c342f23) VM state was updated but update time is null?!
> vm id: 6
> >> INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-2c7d2dce)
> >> (logid:a56a9a8c) Begin cleanup expired async-jobs
> >> INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-2c7d2dce)
> >> (logid:a56a9a8c) End cleanup expired async-jobs
> >> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-
> 383a632c)
> >> (logid:541e9ba5) Found 0 removed accounts to cleanup
> >> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-
> 383a632c)
> >> (logid:541e9ba5) Found 0 disabled accounts to cleanup
> >> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-
> 383a632c)
> >> (logid:541e9ba5) Found 0 inactive domains to cleanup
> >> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-
> 383a632c)
> >> (logid:541e9ba5) Found 0 disabled projects to cleanup
> >> WARN  [c.c.h.x.r.CitrixResourceBase]
> (DirectAgent-16:ctx-7c901443)
> >> 

Re: Modern template hosting

[Will Stevens]

I will be guessing and we will see what I can come up with. :)

On Mar 3, 2017 7:09 PM, "Erik Weber"  wrote:

> Feel free to do it, I wouldn't know where to begin to ask such a
> question tbh :-)
>
> --
> Erik
>
> On Sat, Mar 4, 2017 at 12:51 AM, Will Stevens 
> wrote:
> > Erik, yes I think this is the right approach and covers the main problem
> > case. I can start the conversation with infra unless you would like to.
> Let
> > me know.
> >
> > We did get some statistics from the other mirrors last year, but I don't
> > think we ever had visibility into the cloud.com repository traffic.
> >
> > On Mar 3, 2017 6:42 PM, "Erik Weber"  wrote:
> >
> > It'll most likely only be an issue in the case where a mirror goes
> > down without any immediate chance of getting back up, right?
> > Could we check with ASF Infra beforehand if such case is OK to warrant
> > an urgent Jira ticket to resolve - should it happen?
> >
> > We can document all the mirrors and provide the howtos for manually
> > downloading and seeding the image as a backup.
> >
> > Do we have any traffic statistics for the current solution, or is that
> > not available at S3?
> >
> > --
> > Erik
> >
> > On Sat, Mar 4, 2017 at 12:28 AM, Will Stevens 
> > wrote:
> >> Yes Erik, I agree with you. The only thing that could add complexity in
> >> this is the fact that we don't control the domain, the ASF does (from
> what
> >> I understand).  Do we expect this to be managed by ASF infra or would
> > there
> >> be another way? Ideally the domain is community controlled so we can
> adapt
> >> in case of change without long delays.
> >>
> >> I am 100% in agreement that this is the shortest path and likely to be
> > good
> >> enough.
> >>
> >> On Mar 3, 2017 6:19 PM, "Erik Weber"  wrote:
> >>
> >> IMHO; implement it as simple as possible, use DNS RR and assume/expect
> >> mirror hosts to use the same path.
> >>
> >>
> >> Yes I know; DNS RR isn't bulletproof, but as already mentioned in this
> >> thread - mirrors rarely go down and they aren't used all that often
> >> (primarily seeding).
> >> We can keep a low TTL on the record so that we're able to remove a
> >> mirror that goes down for a significant period of time (more than
> >> hours).
> >>
> >> And yes I know, some operators might find it a burden having to set up
> >> a new vhost with another document root, but tbh if they don't care
> >> enough to do it we'll manage without that mirror.
> >>
> >> --
> >> Erik
> >>
> >> On Sat, Mar 4, 2017 at 12:09 AM, Will Stevens  >
> >> wrote:
> >>> I agree with Paul. Look at the list of things they have to
> >>> learn/master/care about. We don't want to add to that list, we want to
> >>> remove from that list. Make the project/product more accessible. Reduce
> >> the
> >>> barrier to entry...
> >>>
> >>> On Mar 3, 2017 5:50 PM, "Chiradeep Vittal" 
> wrote:
> >>>
>  Seriously?
>  apt-get install apache2
> 
>  To install CloudStack, they need to know
>  - DB installation, perhaps some SQL
>  - VLAN configuration on their switches
>  - Ins-and-outs between port forwarding, static ips,
>  - NFS
>  - package management
>  - VHDs, qcow2. vmdk
>  - hyperviosrs
>  - and on and on.
> 
>  If they can figure all that out and not figure out a web server, they
> > can
>  always come to the mailing list.
> 
> 
> 
> 
>  On Fri, Mar 3, 2017 at 1:45 PM, Paul Angus 
>  wrote:
> 
>  > The issue is not with supporting highly experienced cloud operators.
> > It
>  is
>  > an issue for new users and other 'relatively' inexperienced
> operators.
>  >
>  > I have helped enough newbies and cloud operators who have been
> running
>  > their cloud for a while to know that the barriers to entry are too
> > high
>  as
>  > they are. And telling anyone that they need to create a web server
> so
>  that
>  > they can add their initial template to get started or in order to
> >> create
>  a
>  > new zone, just isn't going to fly.
>  >
>  > I'm a huge advocate of the 'download.cloudstack.org' endpoint which
> > the
>  > community can add/remove mirrors to or from for system VMs or
> built-in
>  > templates.  Can the same system be used for binary repos ? although
> >> there
>  > is an added complication of redist vs no-redist there...
>  >
>  >
>  >
>  >
>  >
>  >
>  > paul.an...@shapeblue.com
>  > www.shapeblue.com
>  > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>  > @shapeblue
>  >
>  >
>  >
>  >
>  > -Original Message-
>  > From: Chiradeep Vittal [mailto:chirade...@gmail.com]
>  > Sent: 03 March 2017 18:28
>  > To: dev 
>  > Subject: 

Re: Modern template hosting

[Erik Weber]

Feel free to do it, I wouldn't know where to begin to ask such a
question tbh :-)

-- 
Erik

On Sat, Mar 4, 2017 at 12:51 AM, Will Stevens  wrote:
> Erik, yes I think this is the right approach and covers the main problem
> case. I can start the conversation with infra unless you would like to. Let
> me know.
>
> We did get some statistics from the other mirrors last year, but I don't
> think we ever had visibility into the cloud.com repository traffic.
>
> On Mar 3, 2017 6:42 PM, "Erik Weber"  wrote:
>
> It'll most likely only be an issue in the case where a mirror goes
> down without any immediate chance of getting back up, right?
> Could we check with ASF Infra beforehand if such case is OK to warrant
> an urgent Jira ticket to resolve - should it happen?
>
> We can document all the mirrors and provide the howtos for manually
> downloading and seeding the image as a backup.
>
> Do we have any traffic statistics for the current solution, or is that
> not available at S3?
>
> --
> Erik
>
> On Sat, Mar 4, 2017 at 12:28 AM, Will Stevens 
> wrote:
>> Yes Erik, I agree with you. The only thing that could add complexity in
>> this is the fact that we don't control the domain, the ASF does (from what
>> I understand).  Do we expect this to be managed by ASF infra or would
> there
>> be another way? Ideally the domain is community controlled so we can adapt
>> in case of change without long delays.
>>
>> I am 100% in agreement that this is the shortest path and likely to be
> good
>> enough.
>>
>> On Mar 3, 2017 6:19 PM, "Erik Weber"  wrote:
>>
>> IMHO; implement it as simple as possible, use DNS RR and assume/expect
>> mirror hosts to use the same path.
>>
>>
>> Yes I know; DNS RR isn't bulletproof, but as already mentioned in this
>> thread - mirrors rarely go down and they aren't used all that often
>> (primarily seeding).
>> We can keep a low TTL on the record so that we're able to remove a
>> mirror that goes down for a significant period of time (more than
>> hours).
>>
>> And yes I know, some operators might find it a burden having to set up
>> a new vhost with another document root, but tbh if they don't care
>> enough to do it we'll manage without that mirror.
>>
>> --
>> Erik
>>
>> On Sat, Mar 4, 2017 at 12:09 AM, Will Stevens 
>> wrote:
>>> I agree with Paul. Look at the list of things they have to
>>> learn/master/care about. We don't want to add to that list, we want to
>>> remove from that list. Make the project/product more accessible. Reduce
>> the
>>> barrier to entry...
>>>
>>> On Mar 3, 2017 5:50 PM, "Chiradeep Vittal"  wrote:
>>>
 Seriously?
 apt-get install apache2

 To install CloudStack, they need to know
 - DB installation, perhaps some SQL
 - VLAN configuration on their switches
 - Ins-and-outs between port forwarding, static ips,
 - NFS
 - package management
 - VHDs, qcow2. vmdk
 - hyperviosrs
 - and on and on.

 If they can figure all that out and not figure out a web server, they
> can
 always come to the mailing list.




 On Fri, Mar 3, 2017 at 1:45 PM, Paul Angus 
 wrote:

 > The issue is not with supporting highly experienced cloud operators.
> It
 is
 > an issue for new users and other 'relatively' inexperienced operators.
 >
 > I have helped enough newbies and cloud operators who have been running
 > their cloud for a while to know that the barriers to entry are too
> high
 as
 > they are. And telling anyone that they need to create a web server so
 that
 > they can add their initial template to get started or in order to
>> create
 a
 > new zone, just isn't going to fly.
 >
 > I'm a huge advocate of the 'download.cloudstack.org' endpoint which
> the
 > community can add/remove mirrors to or from for system VMs or built-in
 > templates.  Can the same system be used for binary repos ? although
>> there
 > is an added complication of redist vs no-redist there...
 >
 >
 >
 >
 >
 >
 > paul.an...@shapeblue.com
 > www.shapeblue.com
 > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
 > @shapeblue
 >
 >
 >
 >
 > -Original Message-
 > From: Chiradeep Vittal [mailto:chirade...@gmail.com]
 > Sent: 03 March 2017 18:28
 > To: dev 
 > Subject: Re: Modern template hosting
 >
 > I do feel like this is early optimization. Mirrors rarely fail. I'd
 expect
 > a single web server hosted on Apache Infra without any monitors to
> fail
 > more often than a mirror. We already expect Wido's systemvm repository
>> to
 > be up all the time. And it has been. Similarly, I don't believe Nux's
 > repository has ever been down. And if Accelerite wants 

Re: Modern template hosting

[Will Stevens]

Erik, yes I think this is the right approach and covers the main problem
case. I can start the conversation with infra unless you would like to. Let
me know.

We did get some statistics from the other mirrors last year, but I don't
think we ever had visibility into the cloud.com repository traffic.

On Mar 3, 2017 6:42 PM, "Erik Weber"  wrote:

It'll most likely only be an issue in the case where a mirror goes
down without any immediate chance of getting back up, right?
Could we check with ASF Infra beforehand if such case is OK to warrant
an urgent Jira ticket to resolve - should it happen?

We can document all the mirrors and provide the howtos for manually
downloading and seeding the image as a backup.

Do we have any traffic statistics for the current solution, or is that
not available at S3?

--
Erik

On Sat, Mar 4, 2017 at 12:28 AM, Will Stevens 
wrote:
> Yes Erik, I agree with you. The only thing that could add complexity in
> this is the fact that we don't control the domain, the ASF does (from what
> I understand).  Do we expect this to be managed by ASF infra or would
there
> be another way? Ideally the domain is community controlled so we can adapt
> in case of change without long delays.
>
> I am 100% in agreement that this is the shortest path and likely to be
good
> enough.
>
> On Mar 3, 2017 6:19 PM, "Erik Weber"  wrote:
>
> IMHO; implement it as simple as possible, use DNS RR and assume/expect
> mirror hosts to use the same path.
>
>
> Yes I know; DNS RR isn't bulletproof, but as already mentioned in this
> thread - mirrors rarely go down and they aren't used all that often
> (primarily seeding).
> We can keep a low TTL on the record so that we're able to remove a
> mirror that goes down for a significant period of time (more than
> hours).
>
> And yes I know, some operators might find it a burden having to set up
> a new vhost with another document root, but tbh if they don't care
> enough to do it we'll manage without that mirror.
>
> --
> Erik
>
> On Sat, Mar 4, 2017 at 12:09 AM, Will Stevens 
> wrote:
>> I agree with Paul. Look at the list of things they have to
>> learn/master/care about. We don't want to add to that list, we want to
>> remove from that list. Make the project/product more accessible. Reduce
> the
>> barrier to entry...
>>
>> On Mar 3, 2017 5:50 PM, "Chiradeep Vittal"  wrote:
>>
>>> Seriously?
>>> apt-get install apache2
>>>
>>> To install CloudStack, they need to know
>>> - DB installation, perhaps some SQL
>>> - VLAN configuration on their switches
>>> - Ins-and-outs between port forwarding, static ips,
>>> - NFS
>>> - package management
>>> - VHDs, qcow2. vmdk
>>> - hyperviosrs
>>> - and on and on.
>>>
>>> If they can figure all that out and not figure out a web server, they
can
>>> always come to the mailing list.
>>>
>>>
>>>
>>>
>>> On Fri, Mar 3, 2017 at 1:45 PM, Paul Angus 
>>> wrote:
>>>
>>> > The issue is not with supporting highly experienced cloud operators.
It
>>> is
>>> > an issue for new users and other 'relatively' inexperienced operators.
>>> >
>>> > I have helped enough newbies and cloud operators who have been running
>>> > their cloud for a while to know that the barriers to entry are too
high
>>> as
>>> > they are. And telling anyone that they need to create a web server so
>>> that
>>> > they can add their initial template to get started or in order to
> create
>>> a
>>> > new zone, just isn't going to fly.
>>> >
>>> > I'm a huge advocate of the 'download.cloudstack.org' endpoint which
the
>>> > community can add/remove mirrors to or from for system VMs or built-in
>>> > templates.  Can the same system be used for binary repos ? although
> there
>>> > is an added complication of redist vs no-redist there...
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > paul.an...@shapeblue.com
>>> > www.shapeblue.com
>>> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>>> > @shapeblue
>>> >
>>> >
>>> >
>>> >
>>> > -Original Message-
>>> > From: Chiradeep Vittal [mailto:chirade...@gmail.com]
>>> > Sent: 03 March 2017 18:28
>>> > To: dev 
>>> > Subject: Re: Modern template hosting
>>> >
>>> > I do feel like this is early optimization. Mirrors rarely fail. I'd
>>> expect
>>> > a single web server hosted on Apache Infra without any monitors to
fail
>>> > more often than a mirror. We already expect Wido's systemvm repository
> to
>>> > be up all the time. And it has been. Similarly, I don't believe Nux's
>>> > repository has ever been down. And if Accelerite wants to host on S3,
>>> that
>>> > one is pretty solid as well.
>>> >
>>> > This is an infrequent operation in a cloud. After the cloud is
> installed,
>>> > the download servers are only needed for a new zone. If we trust the
> user
>>> > to run a cloud, surely he/she can run a web server to serve some
> built-in
>>> > templates. And if 

[GitHub] cloudstack issue #1180: CLOUDSTACK-9112: Deploy VM failing frequently due to...

[cloudmonger]

Github user cloudmonger commented on the issue:

https://github.com/apache/cloudstack/pull/1180
  
### ACS CI BVT Run
 **Sumarry:**
 Build Number 420
 Hypervisor xenserver
 NetworkType Advanced
 Passed=102
 Failed=3
 Skipped=7

_Link to logs Folder (search by build_no):_ 
https://www.dropbox.com/sh/yj3wnzbceo9uef2/AAB6u-Iap-xztdm6jHX9SjPja?dl=0


**Failed tests:**
* test_routers_network_ops.py

 * test_02_isolate_network_FW_PF_default_routes_egress_false Failed

 * test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true Failed

 * test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false Failing since 
2 runs


**Skipped tests:**
test_01_test_vm_volume_snapshot
test_vm_nic_adapter_vmxnet3
test_static_role_account_acls
test_11_ss_nfs_version_on_ssvm
test_nested_virtualization_vmware
test_3d_gpu_support
test_deploy_vgpu_enabled_vm

**Passed test suits:**
test_deploy_vm_with_userdata.py
test_affinity_groups_projects.py
test_portable_publicip.py
test_over_provisioning.py
test_global_settings.py
test_scale_vm.py
test_service_offerings.py
test_routers_iptables_default_policy.py
test_loadbalance.py
test_routers.py
test_reset_vm_on_reboot.py
test_deploy_vms_with_varied_deploymentplanners.py
test_network.py
test_router_dns.py
test_non_contigiousvlan.py
test_login.py
test_deploy_vm_iso.py
test_list_ids_parameter.py
test_public_ip_range.py
test_multipleips_per_nic.py
test_regions.py
test_affinity_groups.py
test_network_acl.py
test_pvlan.py
test_volumes.py
test_nic.py
test_deploy_vm_root_resize.py
test_resource_detail.py
test_secondary_storage.py
test_vm_life_cycle.py
test_disk_offerings.py


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Modern template hosting

[Will Stevens]

I don't think your analogy works. It is difficult to get acs up and
running. You may take it for granted, but I know that struggle has made
some people move on.

I think it is more analogous with not buying a car because the dealer won't
give you the keys until you can prove you can change the oil.

On Mar 3, 2017 6:36 PM, "Chiradeep Vittal"  wrote:

> To me, that is like saying that a customer won't buy a car because he can't
> figure out how to roll down his window, so let's build a voice-controlled
> window controller. Can be done, sounds great, but what happens when that
> voice controller thing in the cloud goes down?
>
>
> On Fri, Mar 3, 2017 at 3:09 PM, Will Stevens 
> wrote:
>
> > I agree with Paul. Look at the list of things they have to
> > learn/master/care about. We don't want to add to that list, we want to
> > remove from that list. Make the project/product more accessible. Reduce
> the
> > barrier to entry...
> >
> > On Mar 3, 2017 5:50 PM, "Chiradeep Vittal"  wrote:
> >
> > > Seriously?
> > > apt-get install apache2
> > >
> > > To install CloudStack, they need to know
> > > - DB installation, perhaps some SQL
> > > - VLAN configuration on their switches
> > > - Ins-and-outs between port forwarding, static ips,
> > > - NFS
> > > - package management
> > > - VHDs, qcow2. vmdk
> > > - hyperviosrs
> > > - and on and on.
> > >
> > > If they can figure all that out and not figure out a web server, they
> can
> > > always come to the mailing list.
> > >
> > >
> > >
> > >
> > > On Fri, Mar 3, 2017 at 1:45 PM, Paul Angus 
> > > wrote:
> > >
> > > > The issue is not with supporting highly experienced cloud operators.
> It
> > > is
> > > > an issue for new users and other 'relatively' inexperienced
> operators.
> > > >
> > > > I have helped enough newbies and cloud operators who have been
> running
> > > > their cloud for a while to know that the barriers to entry are too
> high
> > > as
> > > > they are. And telling anyone that they need to create a web server so
> > > that
> > > > they can add their initial template to get started or in order to
> > create
> > > a
> > > > new zone, just isn't going to fly.
> > > >
> > > > I'm a huge advocate of the 'download.cloudstack.org' endpoint which
> > the
> > > > community can add/remove mirrors to or from for system VMs or
> built-in
> > > > templates.  Can the same system be used for binary repos ? although
> > there
> > > > is an added complication of redist vs no-redist there...
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > paul.an...@shapeblue.com
> > > > www.shapeblue.com
> > > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > > > @shapeblue
> > > >
> > > >
> > > >
> > > >
> > > > -Original Message-
> > > > From: Chiradeep Vittal [mailto:chirade...@gmail.com]
> > > > Sent: 03 March 2017 18:28
> > > > To: dev 
> > > > Subject: Re: Modern template hosting
> > > >
> > > > I do feel like this is early optimization. Mirrors rarely fail. I'd
> > > expect
> > > > a single web server hosted on Apache Infra without any monitors to
> fail
> > > > more often than a mirror. We already expect Wido's systemvm
> repository
> > to
> > > > be up all the time. And it has been. Similarly, I don't believe Nux's
> > > > repository has ever been down. And if Accelerite wants to host on S3,
> > > that
> > > > one is pretty solid as well.
> > > >
> > > > This is an infrequent operation in a cloud. After the cloud is
> > installed,
> > > > the download servers are only needed for a new zone. If we trust the
> > user
> > > > to run a cloud, surely he/she can run a web server to serve some
> > built-in
> > > > templates. And if her cloud is successful, she needs to figure out
> how
> > to
> > > > host her templates anyway and not rely on 3rd parties.
> > > >
> > > >
> > > > On Fri, Mar 3, 2017 at 9:31 AM, Will Stevens 
> > > > wrote:
> > > >
> > > > > 1) If the legacy implementations do not support redirects, that
> does
> > > > > cause a problem.  A potential solution in that case is to have the
> > web
> > > > > server actually proxy the download, but that is not ideal and I
> would
> > > > > like to avoid it if possible.  Thanks for bringing that up
> Chiradeep.
> > > > >
> > > > > 2) I think we need to have a single URL which people can target.
> > Once
> > > > > they make the switch to the new URL, we want the implementation to
> be
> > > > > able to handle mirror failures without affecting the end client.
> We
> > > > > want to avoid the situation where an ACS user will ever have to
> > change
> > > > > this URL more than once.  Mirror failures SHOULD NOT affect the ACS
> > > > > users assuming there is still at least one mirror who can serve the
> > > > requested resource.
> > > > >
> > > > > These are obviously my personal opinions and others will probably
> > have
> > > > > differing opinions.
> > > 

Re: Modern template hosting

[Erik Weber]

It'll most likely only be an issue in the case where a mirror goes
down without any immediate chance of getting back up, right?
Could we check with ASF Infra beforehand if such case is OK to warrant
an urgent Jira ticket to resolve - should it happen?

We can document all the mirrors and provide the howtos for manually
downloading and seeding the image as a backup.

Do we have any traffic statistics for the current solution, or is that
not available at S3?

-- 
Erik

On Sat, Mar 4, 2017 at 12:28 AM, Will Stevens  wrote:
> Yes Erik, I agree with you. The only thing that could add complexity in
> this is the fact that we don't control the domain, the ASF does (from what
> I understand).  Do we expect this to be managed by ASF infra or would there
> be another way? Ideally the domain is community controlled so we can adapt
> in case of change without long delays.
>
> I am 100% in agreement that this is the shortest path and likely to be good
> enough.
>
> On Mar 3, 2017 6:19 PM, "Erik Weber"  wrote:
>
> IMHO; implement it as simple as possible, use DNS RR and assume/expect
> mirror hosts to use the same path.
>
>
> Yes I know; DNS RR isn't bulletproof, but as already mentioned in this
> thread - mirrors rarely go down and they aren't used all that often
> (primarily seeding).
> We can keep a low TTL on the record so that we're able to remove a
> mirror that goes down for a significant period of time (more than
> hours).
>
> And yes I know, some operators might find it a burden having to set up
> a new vhost with another document root, but tbh if they don't care
> enough to do it we'll manage without that mirror.
>
> --
> Erik
>
> On Sat, Mar 4, 2017 at 12:09 AM, Will Stevens 
> wrote:
>> I agree with Paul. Look at the list of things they have to
>> learn/master/care about. We don't want to add to that list, we want to
>> remove from that list. Make the project/product more accessible. Reduce
> the
>> barrier to entry...
>>
>> On Mar 3, 2017 5:50 PM, "Chiradeep Vittal"  wrote:
>>
>>> Seriously?
>>> apt-get install apache2
>>>
>>> To install CloudStack, they need to know
>>> - DB installation, perhaps some SQL
>>> - VLAN configuration on their switches
>>> - Ins-and-outs between port forwarding, static ips,
>>> - NFS
>>> - package management
>>> - VHDs, qcow2. vmdk
>>> - hyperviosrs
>>> - and on and on.
>>>
>>> If they can figure all that out and not figure out a web server, they can
>>> always come to the mailing list.
>>>
>>>
>>>
>>>
>>> On Fri, Mar 3, 2017 at 1:45 PM, Paul Angus 
>>> wrote:
>>>
>>> > The issue is not with supporting highly experienced cloud operators. It
>>> is
>>> > an issue for new users and other 'relatively' inexperienced operators.
>>> >
>>> > I have helped enough newbies and cloud operators who have been running
>>> > their cloud for a while to know that the barriers to entry are too high
>>> as
>>> > they are. And telling anyone that they need to create a web server so
>>> that
>>> > they can add their initial template to get started or in order to
> create
>>> a
>>> > new zone, just isn't going to fly.
>>> >
>>> > I'm a huge advocate of the 'download.cloudstack.org' endpoint which the
>>> > community can add/remove mirrors to or from for system VMs or built-in
>>> > templates.  Can the same system be used for binary repos ? although
> there
>>> > is an added complication of redist vs no-redist there...
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > paul.an...@shapeblue.com
>>> > www.shapeblue.com
>>> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>>> > @shapeblue
>>> >
>>> >
>>> >
>>> >
>>> > -Original Message-
>>> > From: Chiradeep Vittal [mailto:chirade...@gmail.com]
>>> > Sent: 03 March 2017 18:28
>>> > To: dev 
>>> > Subject: Re: Modern template hosting
>>> >
>>> > I do feel like this is early optimization. Mirrors rarely fail. I'd
>>> expect
>>> > a single web server hosted on Apache Infra without any monitors to fail
>>> > more often than a mirror. We already expect Wido's systemvm repository
> to
>>> > be up all the time. And it has been. Similarly, I don't believe Nux's
>>> > repository has ever been down. And if Accelerite wants to host on S3,
>>> that
>>> > one is pretty solid as well.
>>> >
>>> > This is an infrequent operation in a cloud. After the cloud is
> installed,
>>> > the download servers are only needed for a new zone. If we trust the
> user
>>> > to run a cloud, surely he/she can run a web server to serve some
> built-in
>>> > templates. And if her cloud is successful, she needs to figure out how
> to
>>> > host her templates anyway and not rely on 3rd parties.
>>> >
>>> >
>>> > On Fri, Mar 3, 2017 at 9:31 AM, Will Stevens 
>>> > wrote:
>>> >
>>> > > 1) If the legacy implementations do not support redirects, that does
>>> > > cause a problem.  A potential solution in that case is to 

Re: Modern template hosting

[Chiradeep Vittal]

To me, that is like saying that a customer won't buy a car because he can't
figure out how to roll down his window, so let's build a voice-controlled
window controller. Can be done, sounds great, but what happens when that
voice controller thing in the cloud goes down?


On Fri, Mar 3, 2017 at 3:09 PM, Will Stevens 
wrote:

> I agree with Paul. Look at the list of things they have to
> learn/master/care about. We don't want to add to that list, we want to
> remove from that list. Make the project/product more accessible. Reduce the
> barrier to entry...
>
> On Mar 3, 2017 5:50 PM, "Chiradeep Vittal"  wrote:
>
> > Seriously?
> > apt-get install apache2
> >
> > To install CloudStack, they need to know
> > - DB installation, perhaps some SQL
> > - VLAN configuration on their switches
> > - Ins-and-outs between port forwarding, static ips,
> > - NFS
> > - package management
> > - VHDs, qcow2. vmdk
> > - hyperviosrs
> > - and on and on.
> >
> > If they can figure all that out and not figure out a web server, they can
> > always come to the mailing list.
> >
> >
> >
> >
> > On Fri, Mar 3, 2017 at 1:45 PM, Paul Angus 
> > wrote:
> >
> > > The issue is not with supporting highly experienced cloud operators. It
> > is
> > > an issue for new users and other 'relatively' inexperienced operators.
> > >
> > > I have helped enough newbies and cloud operators who have been running
> > > their cloud for a while to know that the barriers to entry are too high
> > as
> > > they are. And telling anyone that they need to create a web server so
> > that
> > > they can add their initial template to get started or in order to
> create
> > a
> > > new zone, just isn't going to fly.
> > >
> > > I'm a huge advocate of the 'download.cloudstack.org' endpoint which
> the
> > > community can add/remove mirrors to or from for system VMs or built-in
> > > templates.  Can the same system be used for binary repos ? although
> there
> > > is an added complication of redist vs no-redist there...
> > >
> > >
> > >
> > >
> > >
> > >
> > > paul.an...@shapeblue.com
> > > www.shapeblue.com
> > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > > @shapeblue
> > >
> > >
> > >
> > >
> > > -Original Message-
> > > From: Chiradeep Vittal [mailto:chirade...@gmail.com]
> > > Sent: 03 March 2017 18:28
> > > To: dev 
> > > Subject: Re: Modern template hosting
> > >
> > > I do feel like this is early optimization. Mirrors rarely fail. I'd
> > expect
> > > a single web server hosted on Apache Infra without any monitors to fail
> > > more often than a mirror. We already expect Wido's systemvm repository
> to
> > > be up all the time. And it has been. Similarly, I don't believe Nux's
> > > repository has ever been down. And if Accelerite wants to host on S3,
> > that
> > > one is pretty solid as well.
> > >
> > > This is an infrequent operation in a cloud. After the cloud is
> installed,
> > > the download servers are only needed for a new zone. If we trust the
> user
> > > to run a cloud, surely he/she can run a web server to serve some
> built-in
> > > templates. And if her cloud is successful, she needs to figure out how
> to
> > > host her templates anyway and not rely on 3rd parties.
> > >
> > >
> > > On Fri, Mar 3, 2017 at 9:31 AM, Will Stevens 
> > > wrote:
> > >
> > > > 1) If the legacy implementations do not support redirects, that does
> > > > cause a problem.  A potential solution in that case is to have the
> web
> > > > server actually proxy the download, but that is not ideal and I would
> > > > like to avoid it if possible.  Thanks for bringing that up Chiradeep.
> > > >
> > > > 2) I think we need to have a single URL which people can target.
> Once
> > > > they make the switch to the new URL, we want the implementation to be
> > > > able to handle mirror failures without affecting the end client.  We
> > > > want to avoid the situation where an ACS user will ever have to
> change
> > > > this URL more than once.  Mirror failures SHOULD NOT affect the ACS
> > > > users assuming there is still at least one mirror who can serve the
> > > requested resource.
> > > >
> > > > These are obviously my personal opinions and others will probably
> have
> > > > differing opinions.
> > > >
> > > > *Will STEVENS*
> > > > Lead Developer
> > > >
> > > > 
> > > >
> > > > On Fri, Mar 3, 2017 at 12:23 PM, Chiradeep Vittal
> > > > 
> > > > wrote:
> > > >
> > > > > 1. If you are targeting legacy installations, they are not able to
> > > > > follow redirects. The line of code that added this capability was
> > > > > added on
> > > > 11/16.
> > > > > 2. If you trust the users to edit the database to change the URL,
> > > > > you can trust them to change it to anything. Just document a known
> > > > > list of good template locations.
> > > > >
> > > > > On Fri, Mar 3, 2017 at 8:13 AM, 

Re: Modern template hosting

[Will Stevens]

Yes Erik, I agree with you. The only thing that could add complexity in
this is the fact that we don't control the domain, the ASF does (from what
I understand).  Do we expect this to be managed by ASF infra or would there
be another way? Ideally the domain is community controlled so we can adapt
in case of change without long delays.

I am 100% in agreement that this is the shortest path and likely to be good
enough.

On Mar 3, 2017 6:19 PM, "Erik Weber"  wrote:

IMHO; implement it as simple as possible, use DNS RR and assume/expect
mirror hosts to use the same path.


Yes I know; DNS RR isn't bulletproof, but as already mentioned in this
thread - mirrors rarely go down and they aren't used all that often
(primarily seeding).
We can keep a low TTL on the record so that we're able to remove a
mirror that goes down for a significant period of time (more than
hours).

And yes I know, some operators might find it a burden having to set up
a new vhost with another document root, but tbh if they don't care
enough to do it we'll manage without that mirror.

--
Erik

On Sat, Mar 4, 2017 at 12:09 AM, Will Stevens 
wrote:
> I agree with Paul. Look at the list of things they have to
> learn/master/care about. We don't want to add to that list, we want to
> remove from that list. Make the project/product more accessible. Reduce
the
> barrier to entry...
>
> On Mar 3, 2017 5:50 PM, "Chiradeep Vittal"  wrote:
>
>> Seriously?
>> apt-get install apache2
>>
>> To install CloudStack, they need to know
>> - DB installation, perhaps some SQL
>> - VLAN configuration on their switches
>> - Ins-and-outs between port forwarding, static ips,
>> - NFS
>> - package management
>> - VHDs, qcow2. vmdk
>> - hyperviosrs
>> - and on and on.
>>
>> If they can figure all that out and not figure out a web server, they can
>> always come to the mailing list.
>>
>>
>>
>>
>> On Fri, Mar 3, 2017 at 1:45 PM, Paul Angus 
>> wrote:
>>
>> > The issue is not with supporting highly experienced cloud operators. It
>> is
>> > an issue for new users and other 'relatively' inexperienced operators.
>> >
>> > I have helped enough newbies and cloud operators who have been running
>> > their cloud for a while to know that the barriers to entry are too high
>> as
>> > they are. And telling anyone that they need to create a web server so
>> that
>> > they can add their initial template to get started or in order to
create
>> a
>> > new zone, just isn't going to fly.
>> >
>> > I'm a huge advocate of the 'download.cloudstack.org' endpoint which the
>> > community can add/remove mirrors to or from for system VMs or built-in
>> > templates.  Can the same system be used for binary repos ? although
there
>> > is an added complication of redist vs no-redist there...
>> >
>> >
>> >
>> >
>> >
>> >
>> > paul.an...@shapeblue.com
>> > www.shapeblue.com
>> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> > @shapeblue
>> >
>> >
>> >
>> >
>> > -Original Message-
>> > From: Chiradeep Vittal [mailto:chirade...@gmail.com]
>> > Sent: 03 March 2017 18:28
>> > To: dev 
>> > Subject: Re: Modern template hosting
>> >
>> > I do feel like this is early optimization. Mirrors rarely fail. I'd
>> expect
>> > a single web server hosted on Apache Infra without any monitors to fail
>> > more often than a mirror. We already expect Wido's systemvm repository
to
>> > be up all the time. And it has been. Similarly, I don't believe Nux's
>> > repository has ever been down. And if Accelerite wants to host on S3,
>> that
>> > one is pretty solid as well.
>> >
>> > This is an infrequent operation in a cloud. After the cloud is
installed,
>> > the download servers are only needed for a new zone. If we trust the
user
>> > to run a cloud, surely he/she can run a web server to serve some
built-in
>> > templates. And if her cloud is successful, she needs to figure out how
to
>> > host her templates anyway and not rely on 3rd parties.
>> >
>> >
>> > On Fri, Mar 3, 2017 at 9:31 AM, Will Stevens 
>> > wrote:
>> >
>> > > 1) If the legacy implementations do not support redirects, that does
>> > > cause a problem.  A potential solution in that case is to have the
web
>> > > server actually proxy the download, but that is not ideal and I would
>> > > like to avoid it if possible.  Thanks for bringing that up Chiradeep.
>> > >
>> > > 2) I think we need to have a single URL which people can target.
Once
>> > > they make the switch to the new URL, we want the implementation to be
>> > > able to handle mirror failures without affecting the end client.  We
>> > > want to avoid the situation where an ACS user will ever have to
change
>> > > this URL more than once.  Mirror failures SHOULD NOT affect the ACS
>> > > users assuming there is still at least one mirror who can serve the
>> > requested resource.
>> > >
>> > > These are obviously my personal 

Re: Modern template hosting

[Erik Weber]

IMHO; implement it as simple as possible, use DNS RR and assume/expect
mirror hosts to use the same path.


Yes I know; DNS RR isn't bulletproof, but as already mentioned in this
thread - mirrors rarely go down and they aren't used all that often
(primarily seeding).
We can keep a low TTL on the record so that we're able to remove a
mirror that goes down for a significant period of time (more than
hours).

And yes I know, some operators might find it a burden having to set up
a new vhost with another document root, but tbh if they don't care
enough to do it we'll manage without that mirror.

-- 
Erik

On Sat, Mar 4, 2017 at 12:09 AM, Will Stevens  wrote:
> I agree with Paul. Look at the list of things they have to
> learn/master/care about. We don't want to add to that list, we want to
> remove from that list. Make the project/product more accessible. Reduce the
> barrier to entry...
>
> On Mar 3, 2017 5:50 PM, "Chiradeep Vittal"  wrote:
>
>> Seriously?
>> apt-get install apache2
>>
>> To install CloudStack, they need to know
>> - DB installation, perhaps some SQL
>> - VLAN configuration on their switches
>> - Ins-and-outs between port forwarding, static ips,
>> - NFS
>> - package management
>> - VHDs, qcow2. vmdk
>> - hyperviosrs
>> - and on and on.
>>
>> If they can figure all that out and not figure out a web server, they can
>> always come to the mailing list.
>>
>>
>>
>>
>> On Fri, Mar 3, 2017 at 1:45 PM, Paul Angus 
>> wrote:
>>
>> > The issue is not with supporting highly experienced cloud operators. It
>> is
>> > an issue for new users and other 'relatively' inexperienced operators.
>> >
>> > I have helped enough newbies and cloud operators who have been running
>> > their cloud for a while to know that the barriers to entry are too high
>> as
>> > they are. And telling anyone that they need to create a web server so
>> that
>> > they can add their initial template to get started or in order to create
>> a
>> > new zone, just isn't going to fly.
>> >
>> > I'm a huge advocate of the 'download.cloudstack.org' endpoint which the
>> > community can add/remove mirrors to or from for system VMs or built-in
>> > templates.  Can the same system be used for binary repos ? although there
>> > is an added complication of redist vs no-redist there...
>> >
>> >
>> >
>> >
>> >
>> >
>> > paul.an...@shapeblue.com
>> > www.shapeblue.com
>> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> > @shapeblue
>> >
>> >
>> >
>> >
>> > -Original Message-
>> > From: Chiradeep Vittal [mailto:chirade...@gmail.com]
>> > Sent: 03 March 2017 18:28
>> > To: dev 
>> > Subject: Re: Modern template hosting
>> >
>> > I do feel like this is early optimization. Mirrors rarely fail. I'd
>> expect
>> > a single web server hosted on Apache Infra without any monitors to fail
>> > more often than a mirror. We already expect Wido's systemvm repository to
>> > be up all the time. And it has been. Similarly, I don't believe Nux's
>> > repository has ever been down. And if Accelerite wants to host on S3,
>> that
>> > one is pretty solid as well.
>> >
>> > This is an infrequent operation in a cloud. After the cloud is installed,
>> > the download servers are only needed for a new zone. If we trust the user
>> > to run a cloud, surely he/she can run a web server to serve some built-in
>> > templates. And if her cloud is successful, she needs to figure out how to
>> > host her templates anyway and not rely on 3rd parties.
>> >
>> >
>> > On Fri, Mar 3, 2017 at 9:31 AM, Will Stevens 
>> > wrote:
>> >
>> > > 1) If the legacy implementations do not support redirects, that does
>> > > cause a problem.  A potential solution in that case is to have the web
>> > > server actually proxy the download, but that is not ideal and I would
>> > > like to avoid it if possible.  Thanks for bringing that up Chiradeep.
>> > >
>> > > 2) I think we need to have a single URL which people can target.  Once
>> > > they make the switch to the new URL, we want the implementation to be
>> > > able to handle mirror failures without affecting the end client.  We
>> > > want to avoid the situation where an ACS user will ever have to change
>> > > this URL more than once.  Mirror failures SHOULD NOT affect the ACS
>> > > users assuming there is still at least one mirror who can serve the
>> > requested resource.
>> > >
>> > > These are obviously my personal opinions and others will probably have
>> > > differing opinions.
>> > >
>> > > *Will STEVENS*
>> > > Lead Developer
>> > >
>> > > 
>> > >
>> > > On Fri, Mar 3, 2017 at 12:23 PM, Chiradeep Vittal
>> > > 
>> > > wrote:
>> > >
>> > > > 1. If you are targeting legacy installations, they are not able to
>> > > > follow redirects. The line of code that added this capability was
>> > > > added on
>> > > 11/16.
>> > > > 2. If you trust the users to 

Re: Modern template hosting

[Will Stevens]

I agree with Paul. Look at the list of things they have to
learn/master/care about. We don't want to add to that list, we want to
remove from that list. Make the project/product more accessible. Reduce the
barrier to entry...

On Mar 3, 2017 5:50 PM, "Chiradeep Vittal"  wrote:

> Seriously?
> apt-get install apache2
>
> To install CloudStack, they need to know
> - DB installation, perhaps some SQL
> - VLAN configuration on their switches
> - Ins-and-outs between port forwarding, static ips,
> - NFS
> - package management
> - VHDs, qcow2. vmdk
> - hyperviosrs
> - and on and on.
>
> If they can figure all that out and not figure out a web server, they can
> always come to the mailing list.
>
>
>
>
> On Fri, Mar 3, 2017 at 1:45 PM, Paul Angus 
> wrote:
>
> > The issue is not with supporting highly experienced cloud operators. It
> is
> > an issue for new users and other 'relatively' inexperienced operators.
> >
> > I have helped enough newbies and cloud operators who have been running
> > their cloud for a while to know that the barriers to entry are too high
> as
> > they are. And telling anyone that they need to create a web server so
> that
> > they can add their initial template to get started or in order to create
> a
> > new zone, just isn't going to fly.
> >
> > I'm a huge advocate of the 'download.cloudstack.org' endpoint which the
> > community can add/remove mirrors to or from for system VMs or built-in
> > templates.  Can the same system be used for binary repos ? although there
> > is an added complication of redist vs no-redist there...
> >
> >
> >
> >
> >
> >
> > paul.an...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> >
> > -Original Message-
> > From: Chiradeep Vittal [mailto:chirade...@gmail.com]
> > Sent: 03 March 2017 18:28
> > To: dev 
> > Subject: Re: Modern template hosting
> >
> > I do feel like this is early optimization. Mirrors rarely fail. I'd
> expect
> > a single web server hosted on Apache Infra without any monitors to fail
> > more often than a mirror. We already expect Wido's systemvm repository to
> > be up all the time. And it has been. Similarly, I don't believe Nux's
> > repository has ever been down. And if Accelerite wants to host on S3,
> that
> > one is pretty solid as well.
> >
> > This is an infrequent operation in a cloud. After the cloud is installed,
> > the download servers are only needed for a new zone. If we trust the user
> > to run a cloud, surely he/she can run a web server to serve some built-in
> > templates. And if her cloud is successful, she needs to figure out how to
> > host her templates anyway and not rely on 3rd parties.
> >
> >
> > On Fri, Mar 3, 2017 at 9:31 AM, Will Stevens 
> > wrote:
> >
> > > 1) If the legacy implementations do not support redirects, that does
> > > cause a problem.  A potential solution in that case is to have the web
> > > server actually proxy the download, but that is not ideal and I would
> > > like to avoid it if possible.  Thanks for bringing that up Chiradeep.
> > >
> > > 2) I think we need to have a single URL which people can target.  Once
> > > they make the switch to the new URL, we want the implementation to be
> > > able to handle mirror failures without affecting the end client.  We
> > > want to avoid the situation where an ACS user will ever have to change
> > > this URL more than once.  Mirror failures SHOULD NOT affect the ACS
> > > users assuming there is still at least one mirror who can serve the
> > requested resource.
> > >
> > > These are obviously my personal opinions and others will probably have
> > > differing opinions.
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > 
> > >
> > > On Fri, Mar 3, 2017 at 12:23 PM, Chiradeep Vittal
> > > 
> > > wrote:
> > >
> > > > 1. If you are targeting legacy installations, they are not able to
> > > > follow redirects. The line of code that added this capability was
> > > > added on
> > > 11/16.
> > > > 2. If you trust the users to edit the database to change the URL,
> > > > you can trust them to change it to anything. Just document a known
> > > > list of good template locations.
> > > >
> > > > On Fri, Mar 3, 2017 at 8:13 AM, Will Stevens 
> > > > wrote:
> > > >
> > > > > So the main issue I see with this is the following.
> > > > >
> > > > > This implementation is designed to target legacy installations
> > > > > which
> > > will
> > > > > be affected when download.cloud.com disappears.  These people will
> > > need
> > > > to
> > > > > make a DB change to replace the 'download.cloud.com' with some
> > > > > other
> > > url
> > > > > (maybe 'download.cloudstack.org').  Once the DB has been updated,
> > > > > we
> > > can
> > > > > not expect anything else of the client, they need to be able to
> > > > 

Re: Modern template hosting

[Chiradeep Vittal]

Seriously?
apt-get install apache2

To install CloudStack, they need to know
- DB installation, perhaps some SQL
- VLAN configuration on their switches
- Ins-and-outs between port forwarding, static ips,
- NFS
- package management
- VHDs, qcow2. vmdk
- hyperviosrs
- and on and on.

If they can figure all that out and not figure out a web server, they can
always come to the mailing list.




On Fri, Mar 3, 2017 at 1:45 PM, Paul Angus  wrote:

> The issue is not with supporting highly experienced cloud operators. It is
> an issue for new users and other 'relatively' inexperienced operators.
>
> I have helped enough newbies and cloud operators who have been running
> their cloud for a while to know that the barriers to entry are too high as
> they are. And telling anyone that they need to create a web server so that
> they can add their initial template to get started or in order to create a
> new zone, just isn't going to fly.
>
> I'm a huge advocate of the 'download.cloudstack.org' endpoint which the
> community can add/remove mirrors to or from for system VMs or built-in
> templates.  Can the same system be used for binary repos ? although there
> is an added complication of redist vs no-redist there...
>
>
>
>
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Chiradeep Vittal [mailto:chirade...@gmail.com]
> Sent: 03 March 2017 18:28
> To: dev 
> Subject: Re: Modern template hosting
>
> I do feel like this is early optimization. Mirrors rarely fail. I'd expect
> a single web server hosted on Apache Infra without any monitors to fail
> more often than a mirror. We already expect Wido's systemvm repository to
> be up all the time. And it has been. Similarly, I don't believe Nux's
> repository has ever been down. And if Accelerite wants to host on S3, that
> one is pretty solid as well.
>
> This is an infrequent operation in a cloud. After the cloud is installed,
> the download servers are only needed for a new zone. If we trust the user
> to run a cloud, surely he/she can run a web server to serve some built-in
> templates. And if her cloud is successful, she needs to figure out how to
> host her templates anyway and not rely on 3rd parties.
>
>
> On Fri, Mar 3, 2017 at 9:31 AM, Will Stevens 
> wrote:
>
> > 1) If the legacy implementations do not support redirects, that does
> > cause a problem.  A potential solution in that case is to have the web
> > server actually proxy the download, but that is not ideal and I would
> > like to avoid it if possible.  Thanks for bringing that up Chiradeep.
> >
> > 2) I think we need to have a single URL which people can target.  Once
> > they make the switch to the new URL, we want the implementation to be
> > able to handle mirror failures without affecting the end client.  We
> > want to avoid the situation where an ACS user will ever have to change
> > this URL more than once.  Mirror failures SHOULD NOT affect the ACS
> > users assuming there is still at least one mirror who can serve the
> requested resource.
> >
> > These are obviously my personal opinions and others will probably have
> > differing opinions.
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > 
> >
> > On Fri, Mar 3, 2017 at 12:23 PM, Chiradeep Vittal
> > 
> > wrote:
> >
> > > 1. If you are targeting legacy installations, they are not able to
> > > follow redirects. The line of code that added this capability was
> > > added on
> > 11/16.
> > > 2. If you trust the users to edit the database to change the URL,
> > > you can trust them to change it to anything. Just document a known
> > > list of good template locations.
> > >
> > > On Fri, Mar 3, 2017 at 8:13 AM, Will Stevens 
> > > wrote:
> > >
> > > > So the main issue I see with this is the following.
> > > >
> > > > This implementation is designed to target legacy installations
> > > > which
> > will
> > > > be affected when download.cloud.com disappears.  These people will
> > need
> > > to
> > > > make a DB change to replace the 'download.cloud.com' with some
> > > > other
> > url
> > > > (maybe 'download.cloudstack.org').  Once the DB has been updated,
> > > > we
> > can
> > > > not expect anything else of the client, they need to be able to
> > > > just continue operation as they were without any need to rebuild
> > > > or upgrade their ACS.
> > > >
> > > > If we try to force the decision to the client, then we break the
> > > > legacy implementations.  Since the SSVM is likely going to be the
> > > > client in
> > some
> > > > cases, and since it does not already have the logic to handle the
> > > > 300 approach correctly, I feel like it is not a viable solution
> > > > for the
> > > legacy
> > > > deployments.
> > > >
> > > > In addition to that.  I don't think the client is in any better

Re: :[VOTE] Apache Cloudstack 4.10.0.0

[Tutkowski, Mike]

-1 (binding)

Per my previous e-mails concerning this issue, here is a ticket I created:

https://issues.apache.org/jira/browse/CLOUDSTACK-9821

We should get someone from the networking side of things to investigate.

Thanks!

On 3/3/17, 1:28 AM, "Rohit Yadav"  wrote:

-1 (binding)


All, I've found an upgrade blocker. Pre 4.6 users are required to seed 4.6 
systemvmtemplate to proceed with the upgrade otherwise upgrade fails, and from 
4.9 upgrade to 4.10 does no check/enforcement that 4.10 based systemvmtemplate 
has been seeded/registered, nor the minimum required systemvmtemplate version 
is changed from 4.6.0 to 4.10.0.


After we have merged the strongswan/java8 PR, I had updated the upgrade 
docs on how to upgrade the systemvmtemplate here:


http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.10/upgrade/upgrade-4.9.html


Using the above, I've tried to fix these issues here, please review and 
merge for RC2:

https://github.com/apache/cloudstack/pull/1983


With above fix, the aim is 
that users only seed the 4.10 systemvmtemplate before upgrade and post-upgrade 
the upgrade paths fix the entries, global setting etc.


Regards.


From: Tutkowski, Mike 
Sent: 02 March 2017 22:39:08
To: dev@cloudstack.apache.org
Subject: Re: :[VOTE] Apache Cloudstack 4.10.0.0

I rolled back to my master branch at 
da66b06e7d562393da2e4b52206943f8bad49d10 and it works.

It appears something that went into after that commit has broken this. It 
looks like this SHA is about two weeks old and that 43 commits have gone into 
master since it.

On 3/2/17, 7:06 AM, "Tutkowski, Mike"  wrote:

According to where the code fails, though, it appears to be a 
networking problem. If I set a breakpoint before the failure and change a 
variable to say that security groups are not being used, then the VM starts.

I think this is a recently introduced problem because I have another 
branch based off of a slightly older version of master and it works fine here.

> On Mar 2, 2017, at 6:51 AM, Pierre-Luc Dion  
wrote:
>
> Hi Mike,
> Try vm with at least 512MB for memory.
>
>> On Mar 1, 2017 15:01, "Tutkowski, Mike"  
wrote:
>>
>> I see the following exception when trying to deploy a user VM in a 
Basic
>> Zone with two XenServer 6.5 hosts in one cluster. My system VMs have 
all
>> deployed properly. The user template gets downloaded fine. I can see 
the
>> user VM begin to start on a XenServer host, then it goes away. We 
then
>> automatically try on the other host. I can see the VM begin to start 
there
>> for a moment, then it goes away.
>>
>> I am just deploying the user VM’s template and root disk to NFS (same
>> place where the template and root disks of my system VMs are).
>>
>> I am using the built-in XenServer CentOS 5.6 (64 bit) template with 1
>> vCPU, 500 MHz, and 256 MB memory.
>>
>> WARN  [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-7:ctx-35aded78)
>> (logid:aab9c320) Expected 1 answers while executing VmDataCommand but
>> received 2
>> WARN  [c.c.v.VirtualMachinePowerStateSyncImpl] 
(DirectAgentCronJob-14:ctx-27fb1ac3)
>> (logid:2c342f23) VM state was updated but update time is null?! vm 
id: 6
>> INFO  [o.a.c.f.j.i.AsyncJobManagerImpl] 
(AsyncJobMgr-Heartbeat-1:ctx-2c7d2dce)
>> (logid:a56a9a8c) Begin cleanup expired async-jobs
>> INFO  [o.a.c.f.j.i.AsyncJobManagerImpl] 
(AsyncJobMgr-Heartbeat-1:ctx-2c7d2dce)
>> (logid:a56a9a8c) End cleanup expired async-jobs
>> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 removed accounts to cleanup
>> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 disabled accounts to cleanup
>> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 inactive domains to cleanup
>> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 disabled projects to cleanup
>> WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-16:ctx-7c901443)
>> (logid:aab9c320) callHostPlugin failed for cmd: 
default_network_rules with
>> args secIps: 0:, vmName: i-2-6-VM, vmID: 6, vmIP: 10.117.40.53, 
vmMAC:
>> 06:b2:f4:00:00:22,  due to There was a failure communicating with the
>> plugin.
>> WARN  [c.c.h.x.r.w.x.CitrixStartCommandWrapper]
  

[GitHub] cloudstack issue #1278: CLOUDSTACK-9198: Virtual router gets deployed in dis...

[GabrielBrascher]

Github user GabrielBrascher commented on the issue:

https://github.com/apache/cloudstack/pull/1278
  
@anshul1886 I would like to raise the point previously discussed by me and 
@rafaelweingartner. I think that we should pay attention if the change of user 
and caller will really do the job. So far I do not see how this PR changes the 
behavior.

Basically this code changes two parameters in startVirtualRouter 
[_callerUser_ and _caller_ when calling startVirtualRouter(router, callerUser, 
caller, routerDeploymentDefinition.getParams())]. However, those parameters are 
only used inside startVirtualRouter when calling the method start(router, user, 
caller, params, null).

```
if (router.getRole() != Role.VIRTUAL_ROUTER || 
!router.getIsRedundantRouter()) {
return start(router, user, caller, params, null);
}
```

The problem is that the method **start** does not use either the _user_ and 
the _caller_ parameters in the overridden implementation (the one that you are 
using).

```
protected DomainRouterVO start(DomainRouterVO router, final User user, 
final Account caller, final Map params, final DeploymentPlan 
planToDeploy)
throws StorageUnavailableException, 
InsufficientCapacityException, ConcurrentOperationException, 
ResourceUnavailableException {
s_logger.debug("Starting router " + router);
try {
_itMgr.advanceStart(router.getUuid(), params, planToDeploy, 
null);
} catch (final OperationTimedoutException e) {
throw new ResourceUnavailableException("Starting router " + 
router + " failed! " + e.toString(), DataCenter.class, 
router.getDataCenterId());
}
if (router.isStopPending()) {
s_logger.info("Clear the stop pending flag of router " + 
router.getHostName() + " after start router successfully!");
router.setStopPending(false);
router = _routerDao.persist(router);
}
// We don't want the failure of VPN Connection affect the status of
// router, so we try to make connection
// only after router start successfully
final Long vpcId = router.getVpcId();
if (vpcId != null) {
_s2sVpnMgr.reconnectDisconnectedVpnByVpc(vpcId);
}
return _routerDao.findById(router.getId());
}
```

Sorry, but I can't see how your code alters the behavior as intended. Can 
you please show that by changing the parameters _user_ and _caller_ you are 
changing the behavior?

Thanks in advance.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

RE: Modern template hosting

[Paul Angus]

The issue is not with supporting highly experienced cloud operators. It is an 
issue for new users and other 'relatively' inexperienced operators.

I have helped enough newbies and cloud operators who have been running their 
cloud for a while to know that the barriers to entry are too high as they are. 
And telling anyone that they need to create a web server so that they can add 
their initial template to get started or in order to create a new zone, just 
isn't going to fly.

I'm a huge advocate of the 'download.cloudstack.org' endpoint which the 
community can add/remove mirrors to or from for system VMs or built-in 
templates.  Can the same system be used for binary repos ? although there is an 
added complication of redist vs no-redist there...




 

paul.an...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


-Original Message-
From: Chiradeep Vittal [mailto:chirade...@gmail.com] 
Sent: 03 March 2017 18:28
To: dev 
Subject: Re: Modern template hosting

I do feel like this is early optimization. Mirrors rarely fail. I'd expect a 
single web server hosted on Apache Infra without any monitors to fail more 
often than a mirror. We already expect Wido's systemvm repository to be up all 
the time. And it has been. Similarly, I don't believe Nux's repository has ever 
been down. And if Accelerite wants to host on S3, that one is pretty solid as 
well.

This is an infrequent operation in a cloud. After the cloud is installed, the 
download servers are only needed for a new zone. If we trust the user to run a 
cloud, surely he/she can run a web server to serve some built-in templates. And 
if her cloud is successful, she needs to figure out how to host her templates 
anyway and not rely on 3rd parties.


On Fri, Mar 3, 2017 at 9:31 AM, Will Stevens  wrote:

> 1) If the legacy implementations do not support redirects, that does 
> cause a problem.  A potential solution in that case is to have the web 
> server actually proxy the download, but that is not ideal and I would 
> like to avoid it if possible.  Thanks for bringing that up Chiradeep.
>
> 2) I think we need to have a single URL which people can target.  Once 
> they make the switch to the new URL, we want the implementation to be 
> able to handle mirror failures without affecting the end client.  We 
> want to avoid the situation where an ACS user will ever have to change 
> this URL more than once.  Mirror failures SHOULD NOT affect the ACS 
> users assuming there is still at least one mirror who can serve the requested 
> resource.
>
> These are obviously my personal opinions and others will probably have 
> differing opinions.
>
> *Will STEVENS*
> Lead Developer
>
> 
>
> On Fri, Mar 3, 2017 at 12:23 PM, Chiradeep Vittal 
> 
> wrote:
>
> > 1. If you are targeting legacy installations, they are not able to 
> > follow redirects. The line of code that added this capability was 
> > added on
> 11/16.
> > 2. If you trust the users to edit the database to change the URL, 
> > you can trust them to change it to anything. Just document a known 
> > list of good template locations.
> >
> > On Fri, Mar 3, 2017 at 8:13 AM, Will Stevens 
> > wrote:
> >
> > > So the main issue I see with this is the following.
> > >
> > > This implementation is designed to target legacy installations 
> > > which
> will
> > > be affected when download.cloud.com disappears.  These people will
> need
> > to
> > > make a DB change to replace the 'download.cloud.com' with some 
> > > other
> url
> > > (maybe 'download.cloudstack.org').  Once the DB has been updated, 
> > > we
> can
> > > not expect anything else of the client, they need to be able to 
> > > just continue operation as they were without any need to rebuild 
> > > or upgrade their ACS.
> > >
> > > If we try to force the decision to the client, then we break the 
> > > legacy implementations.  Since the SSVM is likely going to be the 
> > > client in
> some
> > > cases, and since it does not already have the logic to handle the 
> > > 300 approach correctly, I feel like it is not a viable solution 
> > > for the
> > legacy
> > > deployments.
> > >
> > > In addition to that.  I don't think the client is in any better
> position
> > to
> > > make the mirror decision than I am.  I am likely able to make a 
> > > more 'educated' decision than the client would be able to because 
> > > I can do different tests on the endpoint(s) before making a final 
> > > decision.
> > >
> > > Does this make sense or am I missing something obvious?
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > 
> > >
> > > On Fri, Mar 3, 2017 at 10:48 AM, Rafael Weingärtner < 
> > > rafaelweingart...@gmail.com> wrote:
> > >
> > > > I think I understood you, but I did not understand you.
> > > >
> > > > Let me see if I can create an example 

[GitHub] cloudstack issue #1987: CLOUDSTACK-9814 : Unable to edit a Sub domain, which...

[rafaelweingartner]

Github user rafaelweingartner commented on the issue:

https://github.com/apache/cloudstack/pull/1987
  
@niteshsarda I suggest you adding line 2281 from "managementService" to 
"domainMgr". The only difference between those two methods is this line. 
Assuming that you changed the method calls, and then everything started 
working. 

Then, please delete the 
com.cloud.server.ManagementServerImpl.updateDomain(UpdateDomainCmd), it is not 
used. It is incredible to have a method with ~100 duplicated between classes.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1988: WIP: CLOUDSTACK-9815 intergration of Applicat...

[DaanHoogland]

GitHub user DaanHoogland opened a pull request:

https://github.com/apache/cloudstack/pull/1988

WIP: CLOUDSTACK-9815 intergration of ApplicationClusterService

  this external plugin will be integrated as core plugin. It was originally 
created as plugin on 4.5. bouncycaste needs updating and several integration 
problems are still blocking a merge.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/shapeblue/cloudstack application-clusters

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1988.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1988


commit ad03449239df0d17ef9dfd9bede96081d9e657e4
Author: Daan Hoogland 
Date:   2017-03-03T20:56:53Z

intergration of ContainerClusterService as ApplicationClusterService
  this external plugin will be integrated as core plugin




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1845: CLOUDSTACK-9689: [Hyper-V] Fixed VM console i...

[ramkatru]

Github user ramkatru commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1845#discussion_r104242930
  
--- Diff: 
services/console-proxy-rdp/rdpconsole/src/main/java/streamer/apr/AprSocketSource.java
 ---
@@ -103,44 +103,46 @@ public void poll(boolean block) {
 if (verbose)
 System.out.println("[" + this + "] INFO: Reading data from 
stream.");
 
-// to unblock during reboot
-long startTime = System.currentTimeMillis();
 // FIXME: If pull is destroyed or socket is closed, segfault 
will happen here
-int actualLength = (block) ? // Blocking read
-Socket.recv(socket, buf.data, buf.offset, 
buf.data.length - buf.offset)
-: // Non-blocking read
-Socket.recvt(socket, buf.data, buf.offset, 
buf.data.length - buf.offset, 500);
+int actualLength;
+if(block) {
+// Blocking read
+actualLength = Socket.recv(socket, buf.data, buf.offset, 
buf.data.length - buf.offset);
+} else {
+// non blocking read with 5 seconds timeout
+Socket.timeoutSet(socket, 500);
+actualLength = Socket.recv(socket, buf.data, buf.offset, 
buf.data.length - buf.offset);
--- End diff --

@anshul1886 can you please verify if timeout can be set on a socket that is 
not opened with APR_SO_NONBLOCK flag?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1813: CLOUDSTACK-9604: Root disk resize support for VMware...

[serg38]

Github user serg38 commented on the issue:

https://github.com/apache/cloudstack/pull/1813
  
Still fails but error is different now
Exception: Job failed: {jobprocstatus : 0, created : 
u'2017-03-03T09:20:53+', jobresult : {errorcode : 530, errortext : u'Unable 
to start a VM due to insufficient capacity'}, cmd : 
u'org.apache.cloudstack.api.command.admin.vm.DeployVMCmdByAdmin', userid : 
u'83ba8c54-ffed-11e6-b1e4-067b8a010729', jobstatus : 2, jobid : 
u'2d7de855-e6d8-4b06-83c4-c880e75c6fb1', jobresultcode : 530, jobinstanceid : 

@borisstoyanov It would be awesome if we can get management server log 
around the exception.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1813: CLOUDSTACK-9604: Root disk resize support for VMware...

[blueorangutan]

Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1813
  
Trillian test result (tid-927)
Environment: vmware-55u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 44318 seconds
Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr1813-t927-vmware-55u3.zip
Intermitten failure detected: 
/marvin/tests/smoke/test_deploy_vm_root_resize.py
Intermitten failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermitten failure detected: 
/marvin/tests/smoke/test_routers_network_ops.py
Intermitten failure detected: /marvin/tests/smoke/test_snapshots.py
Intermitten failure detected: /marvin/tests/smoke/test_vm_snapshots.py
Test completed. 45 look ok, 4 have error(s)


Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_01_test_vm_volume_snapshot | `Failure` | 337.78 | test_vm_snapshots.py
test_04_rvpc_privategw_static_routes | `Failure` | 843.48 | 
test_privategw_acl.py
test_02_list_snapshots_with_removed_data_store | `Error` | 81.00 | 
test_snapshots.py
test_02_list_snapshots_with_removed_data_store | `Error` | 86.09 | 
test_snapshots.py
test_00_deploy_vm_root_resize | `Error` | 157.40 | 
test_deploy_vm_root_resize.py
test_01_vpc_site2site_vpn | Success | 376.44 | test_vpc_vpn.py
test_01_vpc_remote_access_vpn | Success | 161.93 | test_vpc_vpn.py
test_01_redundant_vpc_site2site_vpn | Success | 598.35 | test_vpc_vpn.py
test_02_VPC_default_routes | Success | 389.68 | test_vpc_router_nics.py
test_01_VPC_nics_after_destroy | Success | 786.48 | test_vpc_router_nics.py
test_05_rvpc_multi_tiers | Success | 673.21 | test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics | Success | 1549.38 | 
test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | 
Success | 738.76 | test_vpc_redundant.py
test_02_redundant_VPC_default_routes | Success | 660.05 | 
test_vpc_redundant.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | Success | 1376.11 | 
test_vpc_redundant.py
test_09_delete_detached_volume | Success | 31.00 | test_volumes.py
test_06_download_detached_volume | Success | 50.61 | test_volumes.py
test_05_detach_volume | Success | 110.34 | test_volumes.py
test_04_delete_attached_volume | Success | 15.24 | test_volumes.py
test_03_download_attached_volume | Success | 20.31 | test_volumes.py
test_02_attach_volume | Success | 58.76 | test_volumes.py
test_01_create_volume | Success | 524.94 | test_volumes.py
test_change_service_offering_for_vm_with_snapshots | Success | 584.65 | 
test_vm_snapshots.py
test_03_delete_vm_snapshots | Success | 275.23 | test_vm_snapshots.py
test_02_revert_vm_snapshots | Success | 232.19 | test_vm_snapshots.py
test_01_create_vm_snapshots | Success | 161.71 | test_vm_snapshots.py
test_deploy_vm_multiple | Success | 247.59 | test_vm_life_cycle.py
test_deploy_vm | Success | 0.03 | test_vm_life_cycle.py
test_advZoneVirtualRouter | Success | 0.02 | test_vm_life_cycle.py
test_10_attachAndDetach_iso | Success | 26.83 | test_vm_life_cycle.py
test_09_expunge_vm | Success | 125.23 | test_vm_life_cycle.py
test_08_migrate_vm | Success | 66.07 | test_vm_life_cycle.py
test_07_restore_vm | Success | 0.10 | test_vm_life_cycle.py
test_06_destroy_vm | Success | 10.17 | test_vm_life_cycle.py
test_03_reboot_vm | Success | 5.15 | test_vm_life_cycle.py
test_02_start_vm | Success | 20.30 | test_vm_life_cycle.py
test_01_stop_vm | Success | 10.15 | test_vm_life_cycle.py
test_CreateTemplateWithDuplicateName | Success | 216.47 | test_templates.py
test_08_list_system_templates | Success | 0.03 | test_templates.py
test_07_list_public_templates | Success | 0.04 | test_templates.py
test_05_template_permissions | Success | 0.06 | test_templates.py
test_04_extract_template | Success | 10.24 | test_templates.py
test_03_delete_template | Success | 5.14 | test_templates.py
test_02_edit_template | Success | 90.12 | test_templates.py
test_01_create_template | Success | 110.83 | test_templates.py
test_10_destroy_cpvm | Success | 266.91 | test_ssvm.py
test_09_destroy_ssvm | Success | 243.85 | test_ssvm.py
test_08_reboot_cpvm | Success | 156.62 | test_ssvm.py
test_07_reboot_ssvm | Success | 158.51 | test_ssvm.py
test_06_stop_cpvm | Success | 211.98 | test_ssvm.py
test_05_stop_ssvm | Success | 183.83 | test_ssvm.py
test_04_cpvm_internals | Success | 1.26 | test_ssvm.py
test_03_ssvm_internals | Success | 3.46 | test_ssvm.py
test_02_list_cpvm_vm | Success | 0.14 | test_ssvm.py
test_01_list_sec_storage_vm | Success | 0.13 | test_ssvm.py
test_01_snapshot_root_disk | Success | 26.17 | test_snapshots.py
test_04_change_offering_small | Success | 96.81 | test_service_offerings.py
test_03_delete_service_offering 

[GitHub] cloudstack issue #1865: CLOUDSTACK-9705: Unauthenticated API allows Admin pa...

[ramkatru]

Github user ramkatru commented on the issue:

https://github.com/apache/cloudstack/pull/1865
  
tag:mergeready


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1900: CLOUDSTACK-8862: Introduced new state attaching for ...

[ramkatru]

Github user ramkatru commented on the issue:

https://github.com/apache/cloudstack/pull/1900
  
tag:mergeready


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Modern template hosting

[Will Stevens]

yes, system vm templates only.  solve for legacy setups when
download.cloud.com goes away.  optionally, give the community more control
over available mirrors without impacting the documentation and setup
process.

a valid option i think we should still be considering is just trying to get
the ASF to point 'downloads.cloudstack.org' at wido's repository.  at least
we then have basic 'control' over the domain which we are using in the
documentation.

*Will STEVENS*
Lead Developer



On Fri, Mar 3, 2017 at 1:43 PM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> I think so far the mirror scheme we are discussing is not for general
> purpose VMs, right? I was under the impression that we wanted to improve
> the system VMs template management scheme ONLY.
>
> On Fri, Mar 3, 2017 at 1:27 PM, Chiradeep Vittal 
> wrote:
>
> > I do feel like this is early optimization. Mirrors rarely fail. I'd
> expect
> > a single web server hosted on Apache Infra without any monitors to fail
> > more often than a mirror. We already expect Wido's systemvm repository to
> > be up all the time. And it has been. Similarly, I don't believe Nux's
> > repository has ever been down. And if Accelerite wants to host on S3,
> that
> > one is pretty solid as well.
> >
> > This is an infrequent operation in a cloud. After the cloud is installed,
> > the download servers are only needed for a new zone. If we trust the user
> > to run a cloud, surely he/she can run a web server to serve some built-in
> > templates. And if her cloud is successful, she needs to figure out how to
> > host her templates anyway and not rely on 3rd parties.
> >
> >
> > On Fri, Mar 3, 2017 at 9:31 AM, Will Stevens 
> > wrote:
> >
> > > 1) If the legacy implementations do not support redirects, that does
> > cause
> > > a problem.  A potential solution in that case is to have the web server
> > > actually proxy the download, but that is not ideal and I would like to
> > > avoid it if possible.  Thanks for bringing that up Chiradeep.
> > >
> > > 2) I think we need to have a single URL which people can target.  Once
> > they
> > > make the switch to the new URL, we want the implementation to be able
> to
> > > handle mirror failures without affecting the end client.  We want to
> > avoid
> > > the situation where an ACS user will ever have to change this URL more
> > than
> > > once.  Mirror failures SHOULD NOT affect the ACS users assuming there
> is
> > > still at least one mirror who can serve the requested resource.
> > >
> > > These are obviously my personal opinions and others will probably have
> > > differing opinions.
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > 
> > >
> > > On Fri, Mar 3, 2017 at 12:23 PM, Chiradeep Vittal <
> chirade...@gmail.com>
> > > wrote:
> > >
> > > > 1. If you are targeting legacy installations, they are not able to
> > follow
> > > > redirects. The line of code that added this capability was added on
> > > 11/16.
> > > > 2. If you trust the users to edit the database to change the URL, you
> > can
> > > > trust them to change it to anything. Just document a known list of
> good
> > > > template locations.
> > > >
> > > > On Fri, Mar 3, 2017 at 8:13 AM, Will Stevens 
> > > > wrote:
> > > >
> > > > > So the main issue I see with this is the following.
> > > > >
> > > > > This implementation is designed to target legacy installations
> which
> > > will
> > > > > be affected when download.cloud.com disappears.  These people will
> > > need
> > > > to
> > > > > make a DB change to replace the 'download.cloud.com' with some
> other
> > > url
> > > > > (maybe 'download.cloudstack.org').  Once the DB has been updated,
> we
> > > can
> > > > > not expect anything else of the client, they need to be able to
> just
> > > > > continue operation as they were without any need to rebuild or
> > upgrade
> > > > > their ACS.
> > > > >
> > > > > If we try to force the decision to the client, then we break the
> > legacy
> > > > > implementations.  Since the SSVM is likely going to be the client
> in
> > > some
> > > > > cases, and since it does not already have the logic to handle the
> 300
> > > > > approach correctly, I feel like it is not a viable solution for the
> > > > legacy
> > > > > deployments.
> > > > >
> > > > > In addition to that.  I don't think the client is in any better
> > > position
> > > > to
> > > > > make the mirror decision than I am.  I am likely able to make a
> more
> > > > > 'educated' decision than the client would be able to because I can
> do
> > > > > different tests on the endpoint(s) before making a final decision.
> > > > >
> > > > > Does this make sense or am I missing something obvious?
> > > > >
> > > > > *Will STEVENS*
> > > > > Lead Developer
> > > > >
> > > > > 
> > > > >
> > > > > On Fri, Mar 3, 2017 at 10:48 AM, Rafael Weingärtner <
> > > > > 

Re: Modern template hosting

[Rafael Weingärtner]

I think so far the mirror scheme we are discussing is not for general
purpose VMs, right? I was under the impression that we wanted to improve
the system VMs template management scheme ONLY.

On Fri, Mar 3, 2017 at 1:27 PM, Chiradeep Vittal 
wrote:

> I do feel like this is early optimization. Mirrors rarely fail. I'd expect
> a single web server hosted on Apache Infra without any monitors to fail
> more often than a mirror. We already expect Wido's systemvm repository to
> be up all the time. And it has been. Similarly, I don't believe Nux's
> repository has ever been down. And if Accelerite wants to host on S3, that
> one is pretty solid as well.
>
> This is an infrequent operation in a cloud. After the cloud is installed,
> the download servers are only needed for a new zone. If we trust the user
> to run a cloud, surely he/she can run a web server to serve some built-in
> templates. And if her cloud is successful, she needs to figure out how to
> host her templates anyway and not rely on 3rd parties.
>
>
> On Fri, Mar 3, 2017 at 9:31 AM, Will Stevens 
> wrote:
>
> > 1) If the legacy implementations do not support redirects, that does
> cause
> > a problem.  A potential solution in that case is to have the web server
> > actually proxy the download, but that is not ideal and I would like to
> > avoid it if possible.  Thanks for bringing that up Chiradeep.
> >
> > 2) I think we need to have a single URL which people can target.  Once
> they
> > make the switch to the new URL, we want the implementation to be able to
> > handle mirror failures without affecting the end client.  We want to
> avoid
> > the situation where an ACS user will ever have to change this URL more
> than
> > once.  Mirror failures SHOULD NOT affect the ACS users assuming there is
> > still at least one mirror who can serve the requested resource.
> >
> > These are obviously my personal opinions and others will probably have
> > differing opinions.
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > 
> >
> > On Fri, Mar 3, 2017 at 12:23 PM, Chiradeep Vittal 
> > wrote:
> >
> > > 1. If you are targeting legacy installations, they are not able to
> follow
> > > redirects. The line of code that added this capability was added on
> > 11/16.
> > > 2. If you trust the users to edit the database to change the URL, you
> can
> > > trust them to change it to anything. Just document a known list of good
> > > template locations.
> > >
> > > On Fri, Mar 3, 2017 at 8:13 AM, Will Stevens 
> > > wrote:
> > >
> > > > So the main issue I see with this is the following.
> > > >
> > > > This implementation is designed to target legacy installations which
> > will
> > > > be affected when download.cloud.com disappears.  These people will
> > need
> > > to
> > > > make a DB change to replace the 'download.cloud.com' with some other
> > url
> > > > (maybe 'download.cloudstack.org').  Once the DB has been updated, we
> > can
> > > > not expect anything else of the client, they need to be able to just
> > > > continue operation as they were without any need to rebuild or
> upgrade
> > > > their ACS.
> > > >
> > > > If we try to force the decision to the client, then we break the
> legacy
> > > > implementations.  Since the SSVM is likely going to be the client in
> > some
> > > > cases, and since it does not already have the logic to handle the 300
> > > > approach correctly, I feel like it is not a viable solution for the
> > > legacy
> > > > deployments.
> > > >
> > > > In addition to that.  I don't think the client is in any better
> > position
> > > to
> > > > make the mirror decision than I am.  I am likely able to make a more
> > > > 'educated' decision than the client would be able to because I can do
> > > > different tests on the endpoint(s) before making a final decision.
> > > >
> > > > Does this make sense or am I missing something obvious?
> > > >
> > > > *Will STEVENS*
> > > > Lead Developer
> > > >
> > > > 
> > > >
> > > > On Fri, Mar 3, 2017 at 10:48 AM, Rafael Weingärtner <
> > > > rafaelweingart...@gmail.com> wrote:
> > > >
> > > > > I think I understood you, but I did not understand you.
> > > > >
> > > > > Let me see if I can create an example to illustrate how we could do
> > > using
> > > > > the HTTP 300 code.
> > > > >
> > > > > The HTTP 300 code indicates clients that he/she/it has multiple
> > choices
> > > > to
> > > > > access the requested resource. If the request is a HEAD, this is
> the
> > > only
> > > > > thing that the server returns. If the request if a GET, then the
> > server
> > > > > returns the choices for the client. The response type is defined on
> > the
> > > > > server based on the Content-Type, user agent and others.
> > > > >
> > > > > Let’s say the server receives a GET request and informs the
> > > content-type
> > > > as
> > > > > JSON (application/JSON). Your application 

Re: Modern template hosting

[Rafael Weingärtner]

I liked this proposal. I would only question if the ASF Infra would expose
directly the application, or if it would put the application behind a web
server. If we have a web server in front of it, it gets easier to balance
the “proxy” mechanism between multiple instances of this application. I do
not know how is the Infra systems there (do they have a private cloud using
ACS?!), but I would assume they could configure a scale-out mechanism to
replicate instances as soon as the current ones get saturated.

On Fri, Mar 3, 2017 at 12:54 PM, Will Stevens  wrote:

> So now I am leaning towards the following implementation.  Use a query
> parameter (more accessible than a header) to choose between delivery
> options with a sane default if no query param is added.
>
> The following (potential) options would be available:
> a) [ default, no params ] current implementation.  first mirror to answer
> to the web server would handle the request.
> b) [ ?serve=mirrors ] return a 300 with a list of 'valid' mirrors (head
> check on resource path).
> c) [ ?serve=closest ] the closest 'valid' geographic mirror serves the
> request.  falls back to (a) on error.
> d) [ ?serve=proxy ] the web server would use (a) and stream the
> corresponding mirror back to the client.
>
> We would need a documented recommendation for users based on version
> number.  The true legacy would have to use (d). It defaults to a sane
> default which can handle the slow down of a mirror.  It also lays the
> foundation for optimizing (d) if we need to manage the load across the
> mirrors in the case of proxying.
>
> Thoughts?  I feel like we are getting to a point where we understand what
> our potential implementation paths are.
>
> *Will Stevens*
>
> PS - If you see this, you should consider coming to CCC in Miami.  Travel
> assistance closes on March 8th!!!
>
>
>
>
> On Fri, Mar 3, 2017 at 12:31 PM, Will Stevens 
> wrote:
>
> > 1) If the legacy implementations do not support redirects, that does
> cause
> > a problem.  A potential solution in that case is to have the web server
> > actually proxy the download, but that is not ideal and I would like to
> > avoid it if possible.  Thanks for bringing that up Chiradeep.
> >
> > 2) I think we need to have a single URL which people can target.  Once
> > they make the switch to the new URL, we want the implementation to be
> able
> > to handle mirror failures without affecting the end client.  We want to
> > avoid the situation where an ACS user will ever have to change this URL
> > more than once.  Mirror failures SHOULD NOT affect the ACS users assuming
> > there is still at least one mirror who can serve the requested resource.
> >
> > These are obviously my personal opinions and others will probably have
> > differing opinions.
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > 
> >
> > On Fri, Mar 3, 2017 at 12:23 PM, Chiradeep Vittal 
> > wrote:
> >
> >> 1. If you are targeting legacy installations, they are not able to
> follow
> >> redirects. The line of code that added this capability was added on
> 11/16.
> >> 2. If you trust the users to edit the database to change the URL, you
> can
> >> trust them to change it to anything. Just document a known list of good
> >> template locations.
> >>
> >> On Fri, Mar 3, 2017 at 8:13 AM, Will Stevens 
> >> wrote:
> >>
> >> > So the main issue I see with this is the following.
> >> >
> >> > This implementation is designed to target legacy installations which
> >> will
> >> > be affected when download.cloud.com disappears.  These people will
> >> need to
> >> > make a DB change to replace the 'download.cloud.com' with some other
> >> url
> >> > (maybe 'download.cloudstack.org').  Once the DB has been updated, we
> >> can
> >> > not expect anything else of the client, they need to be able to just
> >> > continue operation as they were without any need to rebuild or upgrade
> >> > their ACS.
> >> >
> >> > If we try to force the decision to the client, then we break the
> legacy
> >> > implementations.  Since the SSVM is likely going to be the client in
> >> some
> >> > cases, and since it does not already have the logic to handle the 300
> >> > approach correctly, I feel like it is not a viable solution for the
> >> legacy
> >> > deployments.
> >> >
> >> > In addition to that.  I don't think the client is in any better
> >> position to
> >> > make the mirror decision than I am.  I am likely able to make a more
> >> > 'educated' decision than the client would be able to because I can do
> >> > different tests on the endpoint(s) before making a final decision.
> >> >
> >> > Does this make sense or am I missing something obvious?
> >> >
> >> > *Will STEVENS*
> >> > Lead Developer
> >> >
> >> > 
> >> >
> >> > On Fri, Mar 3, 2017 at 10:48 AM, Rafael Weingärtner <
> >> > rafaelweingart...@gmail.com> wrote:
> >> >
> >> > > I think I 

Re: Modern template hosting

[Chiradeep Vittal]

I do feel like this is early optimization. Mirrors rarely fail. I'd expect
a single web server hosted on Apache Infra without any monitors to fail
more often than a mirror. We already expect Wido's systemvm repository to
be up all the time. And it has been. Similarly, I don't believe Nux's
repository has ever been down. And if Accelerite wants to host on S3, that
one is pretty solid as well.

This is an infrequent operation in a cloud. After the cloud is installed,
the download servers are only needed for a new zone. If we trust the user
to run a cloud, surely he/she can run a web server to serve some built-in
templates. And if her cloud is successful, she needs to figure out how to
host her templates anyway and not rely on 3rd parties.


On Fri, Mar 3, 2017 at 9:31 AM, Will Stevens  wrote:

> 1) If the legacy implementations do not support redirects, that does cause
> a problem.  A potential solution in that case is to have the web server
> actually proxy the download, but that is not ideal and I would like to
> avoid it if possible.  Thanks for bringing that up Chiradeep.
>
> 2) I think we need to have a single URL which people can target.  Once they
> make the switch to the new URL, we want the implementation to be able to
> handle mirror failures without affecting the end client.  We want to avoid
> the situation where an ACS user will ever have to change this URL more than
> once.  Mirror failures SHOULD NOT affect the ACS users assuming there is
> still at least one mirror who can serve the requested resource.
>
> These are obviously my personal opinions and others will probably have
> differing opinions.
>
> *Will STEVENS*
> Lead Developer
>
> 
>
> On Fri, Mar 3, 2017 at 12:23 PM, Chiradeep Vittal 
> wrote:
>
> > 1. If you are targeting legacy installations, they are not able to follow
> > redirects. The line of code that added this capability was added on
> 11/16.
> > 2. If you trust the users to edit the database to change the URL, you can
> > trust them to change it to anything. Just document a known list of good
> > template locations.
> >
> > On Fri, Mar 3, 2017 at 8:13 AM, Will Stevens 
> > wrote:
> >
> > > So the main issue I see with this is the following.
> > >
> > > This implementation is designed to target legacy installations which
> will
> > > be affected when download.cloud.com disappears.  These people will
> need
> > to
> > > make a DB change to replace the 'download.cloud.com' with some other
> url
> > > (maybe 'download.cloudstack.org').  Once the DB has been updated, we
> can
> > > not expect anything else of the client, they need to be able to just
> > > continue operation as they were without any need to rebuild or upgrade
> > > their ACS.
> > >
> > > If we try to force the decision to the client, then we break the legacy
> > > implementations.  Since the SSVM is likely going to be the client in
> some
> > > cases, and since it does not already have the logic to handle the 300
> > > approach correctly, I feel like it is not a viable solution for the
> > legacy
> > > deployments.
> > >
> > > In addition to that.  I don't think the client is in any better
> position
> > to
> > > make the mirror decision than I am.  I am likely able to make a more
> > > 'educated' decision than the client would be able to because I can do
> > > different tests on the endpoint(s) before making a final decision.
> > >
> > > Does this make sense or am I missing something obvious?
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > 
> > >
> > > On Fri, Mar 3, 2017 at 10:48 AM, Rafael Weingärtner <
> > > rafaelweingart...@gmail.com> wrote:
> > >
> > > > I think I understood you, but I did not understand you.
> > > >
> > > > Let me see if I can create an example to illustrate how we could do
> > using
> > > > the HTTP 300 code.
> > > >
> > > > The HTTP 300 code indicates clients that he/she/it has multiple
> choices
> > > to
> > > > access the requested resource. If the request is a HEAD, this is the
> > only
> > > > thing that the server returns. If the request if a GET, then the
> server
> > > > returns the choices for the client. The response type is defined on
> the
> > > > server based on the Content-Type, user agent and others.
> > > >
> > > > Let’s say the server receives a GET request and informs the
> > content-type
> > > as
> > > > JSON (application/JSON). Your application could get the mirrors list,
> > do
> > > > the appending process for the requested resource path, and then
> return
> > a
> > > > JSON array with the possible mirror the client can use. Then, the
> > client
> > > > gets this list and does the processing required to select a mirror.
> > > >
> > > > Now let’s say that a human access the link using a browser. Then, the
> > > > server should receive something like (text/plain) as the
> content-type;
> > I
> > > > think depending on the browser this field may 

Re: Modern template hosting

[Will Stevens]

So now I am leaning towards the following implementation.  Use a query
parameter (more accessible than a header) to choose between delivery
options with a sane default if no query param is added.

The following (potential) options would be available:
a) [ default, no params ] current implementation.  first mirror to answer
to the web server would handle the request.
b) [ ?serve=mirrors ] return a 300 with a list of 'valid' mirrors (head
check on resource path).
c) [ ?serve=closest ] the closest 'valid' geographic mirror serves the
request.  falls back to (a) on error.
d) [ ?serve=proxy ] the web server would use (a) and stream the
corresponding mirror back to the client.

We would need a documented recommendation for users based on version
number.  The true legacy would have to use (d). It defaults to a sane
default which can handle the slow down of a mirror.  It also lays the
foundation for optimizing (d) if we need to manage the load across the
mirrors in the case of proxying.

Thoughts?  I feel like we are getting to a point where we understand what
our potential implementation paths are.

*Will Stevens*

PS - If you see this, you should consider coming to CCC in Miami.  Travel
assistance closes on March 8th!!!




On Fri, Mar 3, 2017 at 12:31 PM, Will Stevens  wrote:

> 1) If the legacy implementations do not support redirects, that does cause
> a problem.  A potential solution in that case is to have the web server
> actually proxy the download, but that is not ideal and I would like to
> avoid it if possible.  Thanks for bringing that up Chiradeep.
>
> 2) I think we need to have a single URL which people can target.  Once
> they make the switch to the new URL, we want the implementation to be able
> to handle mirror failures without affecting the end client.  We want to
> avoid the situation where an ACS user will ever have to change this URL
> more than once.  Mirror failures SHOULD NOT affect the ACS users assuming
> there is still at least one mirror who can serve the requested resource.
>
> These are obviously my personal opinions and others will probably have
> differing opinions.
>
> *Will STEVENS*
> Lead Developer
>
> 
>
> On Fri, Mar 3, 2017 at 12:23 PM, Chiradeep Vittal 
> wrote:
>
>> 1. If you are targeting legacy installations, they are not able to follow
>> redirects. The line of code that added this capability was added on 11/16.
>> 2. If you trust the users to edit the database to change the URL, you can
>> trust them to change it to anything. Just document a known list of good
>> template locations.
>>
>> On Fri, Mar 3, 2017 at 8:13 AM, Will Stevens 
>> wrote:
>>
>> > So the main issue I see with this is the following.
>> >
>> > This implementation is designed to target legacy installations which
>> will
>> > be affected when download.cloud.com disappears.  These people will
>> need to
>> > make a DB change to replace the 'download.cloud.com' with some other
>> url
>> > (maybe 'download.cloudstack.org').  Once the DB has been updated, we
>> can
>> > not expect anything else of the client, they need to be able to just
>> > continue operation as they were without any need to rebuild or upgrade
>> > their ACS.
>> >
>> > If we try to force the decision to the client, then we break the legacy
>> > implementations.  Since the SSVM is likely going to be the client in
>> some
>> > cases, and since it does not already have the logic to handle the 300
>> > approach correctly, I feel like it is not a viable solution for the
>> legacy
>> > deployments.
>> >
>> > In addition to that.  I don't think the client is in any better
>> position to
>> > make the mirror decision than I am.  I am likely able to make a more
>> > 'educated' decision than the client would be able to because I can do
>> > different tests on the endpoint(s) before making a final decision.
>> >
>> > Does this make sense or am I missing something obvious?
>> >
>> > *Will STEVENS*
>> > Lead Developer
>> >
>> > 
>> >
>> > On Fri, Mar 3, 2017 at 10:48 AM, Rafael Weingärtner <
>> > rafaelweingart...@gmail.com> wrote:
>> >
>> > > I think I understood you, but I did not understand you.
>> > >
>> > > Let me see if I can create an example to illustrate how we could do
>> using
>> > > the HTTP 300 code.
>> > >
>> > > The HTTP 300 code indicates clients that he/she/it has multiple
>> choices
>> > to
>> > > access the requested resource. If the request is a HEAD, this is the
>> only
>> > > thing that the server returns. If the request if a GET, then the
>> server
>> > > returns the choices for the client. The response type is defined on
>> the
>> > > server based on the Content-Type, user agent and others.
>> > >
>> > > Let’s say the server receives a GET request and informs the
>> content-type
>> > as
>> > > JSON (application/JSON). Your application could get the mirrors list,
>> do
>> > > the appending process for the requested 

Re: Modern template hosting

[Will Stevens]

1) If the legacy implementations do not support redirects, that does cause
a problem.  A potential solution in that case is to have the web server
actually proxy the download, but that is not ideal and I would like to
avoid it if possible.  Thanks for bringing that up Chiradeep.

2) I think we need to have a single URL which people can target.  Once they
make the switch to the new URL, we want the implementation to be able to
handle mirror failures without affecting the end client.  We want to avoid
the situation where an ACS user will ever have to change this URL more than
once.  Mirror failures SHOULD NOT affect the ACS users assuming there is
still at least one mirror who can serve the requested resource.

These are obviously my personal opinions and others will probably have
differing opinions.

*Will STEVENS*
Lead Developer



On Fri, Mar 3, 2017 at 12:23 PM, Chiradeep Vittal 
wrote:

> 1. If you are targeting legacy installations, they are not able to follow
> redirects. The line of code that added this capability was added on 11/16.
> 2. If you trust the users to edit the database to change the URL, you can
> trust them to change it to anything. Just document a known list of good
> template locations.
>
> On Fri, Mar 3, 2017 at 8:13 AM, Will Stevens 
> wrote:
>
> > So the main issue I see with this is the following.
> >
> > This implementation is designed to target legacy installations which will
> > be affected when download.cloud.com disappears.  These people will need
> to
> > make a DB change to replace the 'download.cloud.com' with some other url
> > (maybe 'download.cloudstack.org').  Once the DB has been updated, we can
> > not expect anything else of the client, they need to be able to just
> > continue operation as they were without any need to rebuild or upgrade
> > their ACS.
> >
> > If we try to force the decision to the client, then we break the legacy
> > implementations.  Since the SSVM is likely going to be the client in some
> > cases, and since it does not already have the logic to handle the 300
> > approach correctly, I feel like it is not a viable solution for the
> legacy
> > deployments.
> >
> > In addition to that.  I don't think the client is in any better position
> to
> > make the mirror decision than I am.  I am likely able to make a more
> > 'educated' decision than the client would be able to because I can do
> > different tests on the endpoint(s) before making a final decision.
> >
> > Does this make sense or am I missing something obvious?
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > 
> >
> > On Fri, Mar 3, 2017 at 10:48 AM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> > > I think I understood you, but I did not understand you.
> > >
> > > Let me see if I can create an example to illustrate how we could do
> using
> > > the HTTP 300 code.
> > >
> > > The HTTP 300 code indicates clients that he/she/it has multiple choices
> > to
> > > access the requested resource. If the request is a HEAD, this is the
> only
> > > thing that the server returns. If the request if a GET, then the server
> > > returns the choices for the client. The response type is defined on the
> > > server based on the Content-Type, user agent and others.
> > >
> > > Let’s say the server receives a GET request and informs the
> content-type
> > as
> > > JSON (application/JSON). Your application could get the mirrors list,
> do
> > > the appending process for the requested resource path, and then return
> a
> > > JSON array with the possible mirror the client can use. Then, the
> client
> > > gets this list and does the processing required to select a mirror.
> > >
> > > Now let’s say that a human access the link using a browser. Then, the
> > > server should receive something like (text/plain) as the content-type;
> I
> > > think depending on the browser this field may differ a little (it would
> > > require some checking). Anyways, the server detects that it is a
> “human”
> > > requesting the resource, then we could serve an HTML page with a
> > Javascript
> > > that uses the mirror list. This Javascript could do some checking and
> > > choose the best mirror for that specific.
> > >
> > > Why do I think this approach is interesting?
> > > The application you developed would be used only to retrieve valid and
> > > trusted sources of system VM images (a trusted repository for mirrors
> > that
> > > we as a community vouch for).  Also, we shift the decision process
> > > regarding mirrors from the server to the client. Then, it is up to
> > clients
> > > to select mirrors, and not up to us (or some of our implementations).
> > >
> > > Did this help?
> > > BTW: I am not saying I am against the way you proposed, which would
> work
> > > fine. It is merely a suggestion using a different perspective.
> > >
> > > On Fri, Mar 3, 2017 at 10:23 AM, Will Stevens <
> williamstev...@gmail.com>

Re: Modern template hosting

[Will Stevens]

ws: Inline...

*Will STEVENS*
Lead Developer



On Fri, Mar 3, 2017 at 12:01 PM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> Yes it makes sense. I agree with you regarding the legacy application. I
> forgot them, my bad. Considering legacy application, I do agree that using
> the HTTP 300 method will not work.
>

​ws: Ya, I think we have to keep this in mind as it is the most pressing
issue that needs to be solved.​

>
> Now regarding the second point in the discussion; I think clients have a
> better position to decide, which mirror is the best for them. The same
> process you do to check the mirror latency, or check if the mirror is
> online could be executed on the client side. As I said before, this shifts
> the responsibility to the client side (this may be a good or a bad thing
> depending on the perspective).
>

​ws: ASSUMING the client scripts a solution similar to what I have done
with the HEAD request on the resource and they pick the one with the lowest
latency, then yes, I agree that is a better solution.  If they are just
randomly picking one of the mirrors returned, then no, I don't think it is
better than what I can provide from the server.

The interesting thing here is that we don't have to choose only one
approach if I am implementing this web server.  We COULD introduce the
concept of using Headers or Query Params to give the client ​a choice of
how they want the mirror to be chosen.  In that case, we would want a
simple fail safe method to be the default and then the more 'advanced'
options to be handled by padding additional information.  This way the
client can choose the 300 approach if the are setup to consume and make an
educated decision about the results.

Something like this:
a) Default: use the current implementation since it handles loaded mirrors
pretty well.
b) Passing a flag would make the server respond with the 300 implementation.
c) Passing a flag would make the server chose the closest geographically
located mirror which successfully responded to the HEAD request.

Does this make sense?

>
> Anyways, I do not think that it would be feasible to use the approach I
> proposed given legacy systems. When I proposed, I did not think of this
> problem.
>
> Besides that comment, I think the system is ok ;)
>
> Before, you also asked opinions about the use of “cloudstack-www” as a repo
> for this mirror list file. I am in favor of that. Then, it would not
> require setting up other repositories and routines to get data from Github
> to a web server at Apache.
>

​ws: Ya, I thought of this while walking home from the train and I think it
makes the most sense.  We already have everything in place to make it work.​

>
> On Fri, Mar 3, 2017 at 11:13 AM, Will Stevens 
> wrote:
>
> > So the main issue I see with this is the following.
> >
> > This implementation is designed to target legacy installations which will
> > be affected when download.cloud.com disappears.  These people will need
> to
> > make a DB change to replace the 'download.cloud.com' with some other url
> > (maybe 'download.cloudstack.org').  Once the DB has been updated, we can
> > not expect anything else of the client, they need to be able to just
> > continue operation as they were without any need to rebuild or upgrade
> > their ACS.
> >
> > If we try to force the decision to the client, then we break the legacy
> > implementations.  Since the SSVM is likely going to be the client in some
> > cases, and since it does not already have the logic to handle the 300
> > approach correctly, I feel like it is not a viable solution for the
> legacy
> > deployments.
> >
> > In addition to that.  I don't think the client is in any better position
> to
> > make the mirror decision than I am.  I am likely able to make a more
> > 'educated' decision than the client would be able to because I can do
> > different tests on the endpoint(s) before making a final decision.
> >
> > Does this make sense or am I missing something obvious?
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > 
> >
> > On Fri, Mar 3, 2017 at 10:48 AM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> > > I think I understood you, but I did not understand you.
> > >
> > > Let me see if I can create an example to illustrate how we could do
> using
> > > the HTTP 300 code.
> > >
> > > The HTTP 300 code indicates clients that he/she/it has multiple choices
> > to
> > > access the requested resource. If the request is a HEAD, this is the
> only
> > > thing that the server returns. If the request if a GET, then the server
> > > returns the choices for the client. The response type is defined on the
> > > server based on the Content-Type, user agent and others.
> > >
> > > Let’s say the server receives a GET request and informs the
> content-type
> > as
> > > JSON (application/JSON). Your application could get the mirrors list,
> do
> > > the appending process 

Re: Modern template hosting

[Chiradeep Vittal]

1. If you are targeting legacy installations, they are not able to follow
redirects. The line of code that added this capability was added on 11/16.
2. If you trust the users to edit the database to change the URL, you can
trust them to change it to anything. Just document a known list of good
template locations.

On Fri, Mar 3, 2017 at 8:13 AM, Will Stevens  wrote:

> So the main issue I see with this is the following.
>
> This implementation is designed to target legacy installations which will
> be affected when download.cloud.com disappears.  These people will need to
> make a DB change to replace the 'download.cloud.com' with some other url
> (maybe 'download.cloudstack.org').  Once the DB has been updated, we can
> not expect anything else of the client, they need to be able to just
> continue operation as they were without any need to rebuild or upgrade
> their ACS.
>
> If we try to force the decision to the client, then we break the legacy
> implementations.  Since the SSVM is likely going to be the client in some
> cases, and since it does not already have the logic to handle the 300
> approach correctly, I feel like it is not a viable solution for the legacy
> deployments.
>
> In addition to that.  I don't think the client is in any better position to
> make the mirror decision than I am.  I am likely able to make a more
> 'educated' decision than the client would be able to because I can do
> different tests on the endpoint(s) before making a final decision.
>
> Does this make sense or am I missing something obvious?
>
> *Will STEVENS*
> Lead Developer
>
> 
>
> On Fri, Mar 3, 2017 at 10:48 AM, Rafael Weingärtner <
> rafaelweingart...@gmail.com> wrote:
>
> > I think I understood you, but I did not understand you.
> >
> > Let me see if I can create an example to illustrate how we could do using
> > the HTTP 300 code.
> >
> > The HTTP 300 code indicates clients that he/she/it has multiple choices
> to
> > access the requested resource. If the request is a HEAD, this is the only
> > thing that the server returns. If the request if a GET, then the server
> > returns the choices for the client. The response type is defined on the
> > server based on the Content-Type, user agent and others.
> >
> > Let’s say the server receives a GET request and informs the content-type
> as
> > JSON (application/JSON). Your application could get the mirrors list, do
> > the appending process for the requested resource path, and then return a
> > JSON array with the possible mirror the client can use. Then, the client
> > gets this list and does the processing required to select a mirror.
> >
> > Now let’s say that a human access the link using a browser. Then, the
> > server should receive something like (text/plain) as the content-type; I
> > think depending on the browser this field may differ a little (it would
> > require some checking). Anyways, the server detects that it is a “human”
> > requesting the resource, then we could serve an HTML page with a
> Javascript
> > that uses the mirror list. This Javascript could do some checking and
> > choose the best mirror for that specific.
> >
> > Why do I think this approach is interesting?
> > The application you developed would be used only to retrieve valid and
> > trusted sources of system VM images (a trusted repository for mirrors
> that
> > we as a community vouch for).  Also, we shift the decision process
> > regarding mirrors from the server to the client. Then, it is up to
> clients
> > to select mirrors, and not up to us (or some of our implementations).
> >
> > Did this help?
> > BTW: I am not saying I am against the way you proposed, which would work
> > fine. It is merely a suggestion using a different perspective.
> >
> > On Fri, Mar 3, 2017 at 10:23 AM, Will Stevens 
> > wrote:
> >
> > > @rafael: in general, I don't think the client should ever be making a
> > > choice about a mirror. We have to assume we are working with a scripted
> > > application and anything hard coded in that implementation on the
> client
> > > side is a risk.
> > >
> > > I may not be understanding the use of the 300 approach, so I will
> > research
> > > it to see if I can make it fit.
> > >
> > > On Mar 3, 2017 9:59 AM, "Rafael Weingärtner" <
> > rafaelweingart...@gmail.com>
> > > wrote:
> > >
> > > Will great job.
> > >
> > > I had the same doubt as Daan.
> > >
> > > Have you considered using HTTP 300 working mode? Then, we could let the
> > > client decide which mirror is the best(closest?). Section “10.3.1” of
> [1]
> > > talks about it; this would put the pressure on deciding upon a mirror
> on
> > > the client side, but I think it would be fair.
> > >
> > > [1] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
> > >
> > > PS: You really like very short variables names!
> > >
> > > On Fri, Mar 3, 2017 at 1:24 AM, Daan Hoogland <
> > daan.hoogl...@shapeblue.com
> > > >
> > > wrote:
> > >
> > > > 

[GitHub] cloudstack issue #1246: CLOUDSTACK-9165 unable to use reserved IP range in a...

[cloudmonger]

Github user cloudmonger commented on the issue:

https://github.com/apache/cloudstack/pull/1246
  
### ACS CI BVT Run
 **Sumarry:**
 Build Number 419
 Hypervisor xenserver
 NetworkType Advanced
 Passed=103
 Failed=2
 Skipped=7

_Link to logs Folder (search by build_no):_ 
https://www.dropbox.com/sh/yj3wnzbceo9uef2/AAB6u-Iap-xztdm6jHX9SjPja?dl=0


**Failed tests:**
* test_volumes.py

 * test_06_download_detached_volume Failed

* test_routers_network_ops.py

 * test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false Failed


**Skipped tests:**
test_01_test_vm_volume_snapshot
test_vm_nic_adapter_vmxnet3
test_static_role_account_acls
test_11_ss_nfs_version_on_ssvm
test_nested_virtualization_vmware
test_3d_gpu_support
test_deploy_vgpu_enabled_vm

**Passed test suits:**
test_deploy_vm_with_userdata.py
test_affinity_groups_projects.py
test_portable_publicip.py
test_over_provisioning.py
test_global_settings.py
test_scale_vm.py
test_service_offerings.py
test_routers_iptables_default_policy.py
test_loadbalance.py
test_routers.py
test_reset_vm_on_reboot.py
test_deploy_vms_with_varied_deploymentplanners.py
test_network.py
test_router_dns.py
test_non_contigiousvlan.py
test_login.py
test_deploy_vm_iso.py
test_list_ids_parameter.py
test_public_ip_range.py
test_multipleips_per_nic.py
test_regions.py
test_affinity_groups.py
test_network_acl.py
test_pvlan.py
test_nic.py
test_deploy_vm_root_resize.py
test_resource_detail.py
test_secondary_storage.py
test_vm_life_cycle.py
test_disk_offerings.py


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1961: Fix for test_snapshots.py using nfs2 instead of nfs ...

[serg38]

Github user serg38 commented on the issue:

https://github.com/apache/cloudstack/pull/1961
  
@rhtyd @borisstoyanov did you have a chance to add nfs2 test data in B.O. ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1961: Fix for test_snapshots.py using nfs2 instead of nfs ...

[blueorangutan]

Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1961
  
Trillian test result (tid-926)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 30702 seconds
Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr1961-t926-kvm-centos7.zip
Intermitten failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermitten failure detected: /marvin/tests/smoke/test_snapshots.py
Intermitten failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Test completed. 47 look ok, 2 have error(s)


Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_04_rvpc_privategw_static_routes | `Failure` | 310.80 | 
test_privategw_acl.py
test_02_list_snapshots_with_removed_data_store | `Error` | 5.42 | 
test_snapshots.py
test_01_vpc_site2site_vpn | Success | 165.41 | test_vpc_vpn.py
test_01_vpc_remote_access_vpn | Success | 66.15 | test_vpc_vpn.py
test_01_redundant_vpc_site2site_vpn | Success | 220.66 | test_vpc_vpn.py
test_02_VPC_default_routes | Success | 265.19 | test_vpc_router_nics.py
test_01_VPC_nics_after_destroy | Success | 481.12 | test_vpc_router_nics.py
test_05_rvpc_multi_tiers | Success | 500.75 | test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics | Success | 1393.39 | 
test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | 
Success | 543.17 | test_vpc_redundant.py
test_02_redundant_VPC_default_routes | Success | 755.80 | 
test_vpc_redundant.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | Success | 1274.49 | 
test_vpc_redundant.py
test_09_delete_detached_volume | Success | 156.61 | test_volumes.py
test_08_resize_volume | Success | 156.35 | test_volumes.py
test_07_resize_fail | Success | 161.45 | test_volumes.py
test_06_download_detached_volume | Success | 156.25 | test_volumes.py
test_05_detach_volume | Success | 155.78 | test_volumes.py
test_04_delete_attached_volume | Success | 146.20 | test_volumes.py
test_03_download_attached_volume | Success | 151.27 | test_volumes.py
test_02_attach_volume | Success | 96.25 | test_volumes.py
test_01_create_volume | Success | 711.30 | test_volumes.py
test_03_delete_vm_snapshots | Success | 275.17 | test_vm_snapshots.py
test_02_revert_vm_snapshots | Success | 95.73 | test_vm_snapshots.py
test_01_create_vm_snapshots | Success | 164.79 | test_vm_snapshots.py
test_deploy_vm_multiple | Success | 252.83 | test_vm_life_cycle.py
test_deploy_vm | Success | 0.03 | test_vm_life_cycle.py
test_advZoneVirtualRouter | Success | 0.02 | test_vm_life_cycle.py
test_10_attachAndDetach_iso | Success | 26.53 | test_vm_life_cycle.py
test_09_expunge_vm | Success | 185.30 | test_vm_life_cycle.py
test_08_migrate_vm | Success | 45.94 | test_vm_life_cycle.py
test_07_restore_vm | Success | 0.12 | test_vm_life_cycle.py
test_06_destroy_vm | Success | 125.80 | test_vm_life_cycle.py
test_03_reboot_vm | Success | 125.82 | test_vm_life_cycle.py
test_02_start_vm | Success | 10.17 | test_vm_life_cycle.py
test_01_stop_vm | Success | 35.30 | test_vm_life_cycle.py
test_CreateTemplateWithDuplicateName | Success | 40.52 | test_templates.py
test_08_list_system_templates | Success | 0.03 | test_templates.py
test_07_list_public_templates | Success | 0.04 | test_templates.py
test_05_template_permissions | Success | 0.05 | test_templates.py
test_04_extract_template | Success | 5.12 | test_templates.py
test_03_delete_template | Success | 5.11 | test_templates.py
test_02_edit_template | Success | 90.18 | test_templates.py
test_01_create_template | Success | 45.45 | test_templates.py
test_10_destroy_cpvm | Success | 161.36 | test_ssvm.py
test_09_destroy_ssvm | Success | 133.13 | test_ssvm.py
test_08_reboot_cpvm | Success | 101.33 | test_ssvm.py
test_07_reboot_ssvm | Success | 103.05 | test_ssvm.py
test_06_stop_cpvm | Success | 131.46 | test_ssvm.py
test_05_stop_ssvm | Success | 133.25 | test_ssvm.py
test_04_cpvm_internals | Success | 0.95 | test_ssvm.py
test_03_ssvm_internals | Success | 2.87 | test_ssvm.py
test_02_list_cpvm_vm | Success | 0.13 | test_ssvm.py
test_01_list_sec_storage_vm | Success | 0.13 | test_ssvm.py
test_01_snapshot_root_disk | Success | 11.12 | test_snapshots.py
test_04_change_offering_small | Success | 210.49 | test_service_offerings.py
test_03_delete_service_offering | Success | 0.04 | test_service_offerings.py
test_02_edit_service_offering | Success | 0.05 | test_service_offerings.py
test_01_create_service_offering | Success | 0.10 | test_service_offerings.py
test_02_sys_template_ready | Success | 0.12 | test_secondary_storage.py
test_01_sys_vm_start | Success | 0.17 | test_secondary_storage.py
test_09_reboot_router | Success | 35.31 | 

Re: Modern template hosting

[Rafael Weingärtner]

Yes it makes sense. I agree with you regarding the legacy application. I
forgot them, my bad. Considering legacy application, I do agree that using
the HTTP 300 method will not work.

Now regarding the second point in the discussion; I think clients have a
better position to decide, which mirror is the best for them. The same
process you do to check the mirror latency, or check if the mirror is
online could be executed on the client side. As I said before, this shifts
the responsibility to the client side (this may be a good or a bad thing
depending on the perspective).

Anyways, I do not think that it would be feasible to use the approach I
proposed given legacy systems. When I proposed, I did not think of this
problem.

Besides that comment, I think the system is ok ;)

Before, you also asked opinions about the use of “cloudstack-www” as a repo
for this mirror list file. I am in favor of that. Then, it would not
require setting up other repositories and routines to get data from Github
to a web server at Apache.

On Fri, Mar 3, 2017 at 11:13 AM, Will Stevens  wrote:

> So the main issue I see with this is the following.
>
> This implementation is designed to target legacy installations which will
> be affected when download.cloud.com disappears.  These people will need to
> make a DB change to replace the 'download.cloud.com' with some other url
> (maybe 'download.cloudstack.org').  Once the DB has been updated, we can
> not expect anything else of the client, they need to be able to just
> continue operation as they were without any need to rebuild or upgrade
> their ACS.
>
> If we try to force the decision to the client, then we break the legacy
> implementations.  Since the SSVM is likely going to be the client in some
> cases, and since it does not already have the logic to handle the 300
> approach correctly, I feel like it is not a viable solution for the legacy
> deployments.
>
> In addition to that.  I don't think the client is in any better position to
> make the mirror decision than I am.  I am likely able to make a more
> 'educated' decision than the client would be able to because I can do
> different tests on the endpoint(s) before making a final decision.
>
> Does this make sense or am I missing something obvious?
>
> *Will STEVENS*
> Lead Developer
>
> 
>
> On Fri, Mar 3, 2017 at 10:48 AM, Rafael Weingärtner <
> rafaelweingart...@gmail.com> wrote:
>
> > I think I understood you, but I did not understand you.
> >
> > Let me see if I can create an example to illustrate how we could do using
> > the HTTP 300 code.
> >
> > The HTTP 300 code indicates clients that he/she/it has multiple choices
> to
> > access the requested resource. If the request is a HEAD, this is the only
> > thing that the server returns. If the request if a GET, then the server
> > returns the choices for the client. The response type is defined on the
> > server based on the Content-Type, user agent and others.
> >
> > Let’s say the server receives a GET request and informs the content-type
> as
> > JSON (application/JSON). Your application could get the mirrors list, do
> > the appending process for the requested resource path, and then return a
> > JSON array with the possible mirror the client can use. Then, the client
> > gets this list and does the processing required to select a mirror.
> >
> > Now let’s say that a human access the link using a browser. Then, the
> > server should receive something like (text/plain) as the content-type; I
> > think depending on the browser this field may differ a little (it would
> > require some checking). Anyways, the server detects that it is a “human”
> > requesting the resource, then we could serve an HTML page with a
> Javascript
> > that uses the mirror list. This Javascript could do some checking and
> > choose the best mirror for that specific.
> >
> > Why do I think this approach is interesting?
> > The application you developed would be used only to retrieve valid and
> > trusted sources of system VM images (a trusted repository for mirrors
> that
> > we as a community vouch for).  Also, we shift the decision process
> > regarding mirrors from the server to the client. Then, it is up to
> clients
> > to select mirrors, and not up to us (or some of our implementations).
> >
> > Did this help?
> > BTW: I am not saying I am against the way you proposed, which would work
> > fine. It is merely a suggestion using a different perspective.
> >
> > On Fri, Mar 3, 2017 at 10:23 AM, Will Stevens 
> > wrote:
> >
> > > @rafael: in general, I don't think the client should ever be making a
> > > choice about a mirror. We have to assume we are working with a scripted
> > > application and anything hard coded in that implementation on the
> client
> > > side is a risk.
> > >
> > > I may not be understanding the use of the 300 approach, so I will
> > research
> > > it to see if I can make it fit.
> > >
> > > On Mar 

Re: Some Jira tickets clean up

[sachin patil]

Thanks @Rafael, That would really help.

On Fri, Mar 3, 2017 at 8:57 PM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> Tickets closed.
>
> Depending on the tickets, some I marked as closed, others as resolved. I
> also configured the status of some to invalid, others to incomplete, others
> as fixed.
>
> On Wed, Mar 1, 2017 at 5:07 PM, Rafael Weingärtner <
> rafaelweingart...@gmail.com> wrote:
>
> > Volks,
> >
> > It has been almost a week. I am assuming that no one knows about these
> > tickets.
> >
> > In the last years, our workflow has been improved greatly; most of the
> > PRs I see have Jira ticket associated and everything seems to be getting
> > better by the time. However, we have some (a lot?) Jira tickets that seem
> > to be left behind. I think a cleanup from time to time is a good thing.
> >
> >
> > In 48 ours I will close all of the tickets mentioned here. If I close
> > something that should not have been closed, please pardon me ;).
> >
> >
> >
> > After this, I will proceed with a new batch of tickets. Hopefully, in
> some
> > iterations of this process, we can get rid of these forgotten tickets.
> >
> > On Thu, Feb 23, 2017 at 3:16 PM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> >> Thanks ;)
> >>
> >> On Thu, Feb 23, 2017 at 3:14 PM, Tutkowski, Mike <
> >> mike.tutkow...@netapp.com> wrote:
> >>
> >>> Thanks, Rafael. I uploaded an ISO and walked through the necessary
> parts
> >>> of the wizard. I agree with you that it has been fixed. I commented on
> the
> >>> ticket, then resolved and closed it just now.
> >>>
> >>> On 2/23/17, 12:46 PM, "Rafael Weingärtner" <
> rafaelweingart...@gmail.com>
> >>> wrote:
> >>>
> >>> Thanks for taking time to look at this Mike.
> >>> The ticket (CLOUDSTACK-4164
> >>> ) I managed
> >>> to
> >>> check, and I believe it can be closed.
> >>>
> >>> On Thu, Feb 23, 2017 at 2:39 PM, Tutkowski, Mike <
> >>> mike.tutkow...@netapp.com>
> >>> wrote:
> >>>
> >>> > Thanks for compiling this list, Rafael!
> >>> >
> >>> > I see I was the reporter for this issue:
> >>> >
> >>> > https://issues.apache.org/jira/browse/CLOUDSTACK-4164
> >>> >
> >>> > I plan to take a look at it and see what the current state is.
> >>> >
> >>> > On 2/23/17, 10:26 AM, "Rafael Weingärtner" <
> >>> rafaelweingart...@gmail.com>
> >>> > wrote:
> >>> >
> >>> > Hi folks,
> >>> > I have been reviewing old Jira tickets and we have a bunch
> >>> that can be
> >>> > closed. The following are the ones I have gone through.
> Before
> >>> I close
> >>> > any
> >>> > ticket I would like to check them with you, especially the
> >>> ones from a
> >>> > category called “Seems to be fixed has to be checked”. I will
> >>> be
> >>> > waiting
> >>> > for your feedback before I proceed to close any ticket.
> >>> >
> >>> > - Have a title that does say much and the ticket does not
> have
> >>> a
> >>> > detailed
> >>> > description
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-2225
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-917
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-947
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-1259
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-1441
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-1576
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-2705
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-5793
> >>> >
> >>> >
> >>> > - Already solved
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-2532
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-6735
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-6837
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-7161
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-8202
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-8781
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-8817
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-9065
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-1069
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-1460
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-1702
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-1870
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-2075
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-2213
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-2439
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-6009
> >>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-5219
> 

Re: Modern template hosting

[Will Stevens]

So the main issue I see with this is the following.

This implementation is designed to target legacy installations which will
be affected when download.cloud.com disappears.  These people will need to
make a DB change to replace the 'download.cloud.com' with some other url
(maybe 'download.cloudstack.org').  Once the DB has been updated, we can
not expect anything else of the client, they need to be able to just
continue operation as they were without any need to rebuild or upgrade
their ACS.

If we try to force the decision to the client, then we break the legacy
implementations.  Since the SSVM is likely going to be the client in some
cases, and since it does not already have the logic to handle the 300
approach correctly, I feel like it is not a viable solution for the legacy
deployments.

In addition to that.  I don't think the client is in any better position to
make the mirror decision than I am.  I am likely able to make a more
'educated' decision than the client would be able to because I can do
different tests on the endpoint(s) before making a final decision.

Does this make sense or am I missing something obvious?

*Will STEVENS*
Lead Developer



On Fri, Mar 3, 2017 at 10:48 AM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> I think I understood you, but I did not understand you.
>
> Let me see if I can create an example to illustrate how we could do using
> the HTTP 300 code.
>
> The HTTP 300 code indicates clients that he/she/it has multiple choices to
> access the requested resource. If the request is a HEAD, this is the only
> thing that the server returns. If the request if a GET, then the server
> returns the choices for the client. The response type is defined on the
> server based on the Content-Type, user agent and others.
>
> Let’s say the server receives a GET request and informs the content-type as
> JSON (application/JSON). Your application could get the mirrors list, do
> the appending process for the requested resource path, and then return a
> JSON array with the possible mirror the client can use. Then, the client
> gets this list and does the processing required to select a mirror.
>
> Now let’s say that a human access the link using a browser. Then, the
> server should receive something like (text/plain) as the content-type; I
> think depending on the browser this field may differ a little (it would
> require some checking). Anyways, the server detects that it is a “human”
> requesting the resource, then we could serve an HTML page with a Javascript
> that uses the mirror list. This Javascript could do some checking and
> choose the best mirror for that specific.
>
> Why do I think this approach is interesting?
> The application you developed would be used only to retrieve valid and
> trusted sources of system VM images (a trusted repository for mirrors that
> we as a community vouch for).  Also, we shift the decision process
> regarding mirrors from the server to the client. Then, it is up to clients
> to select mirrors, and not up to us (or some of our implementations).
>
> Did this help?
> BTW: I am not saying I am against the way you proposed, which would work
> fine. It is merely a suggestion using a different perspective.
>
> On Fri, Mar 3, 2017 at 10:23 AM, Will Stevens 
> wrote:
>
> > @rafael: in general, I don't think the client should ever be making a
> > choice about a mirror. We have to assume we are working with a scripted
> > application and anything hard coded in that implementation on the client
> > side is a risk.
> >
> > I may not be understanding the use of the 300 approach, so I will
> research
> > it to see if I can make it fit.
> >
> > On Mar 3, 2017 9:59 AM, "Rafael Weingärtner" <
> rafaelweingart...@gmail.com>
> > wrote:
> >
> > Will great job.
> >
> > I had the same doubt as Daan.
> >
> > Have you considered using HTTP 300 working mode? Then, we could let the
> > client decide which mirror is the best(closest?). Section “10.3.1” of [1]
> > talks about it; this would put the pressure on deciding upon a mirror on
> > the client side, but I think it would be fair.
> >
> > [1] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
> >
> > PS: You really like very short variables names!
> >
> > On Fri, Mar 3, 2017 at 1:24 AM, Daan Hoogland <
> daan.hoogl...@shapeblue.com
> > >
> > wrote:
> >
> > > Nice little thing Will,
> > >
> > > One question: if I read the code correctly it ‘go’es and tries all
> > mirrors
> > > at once and whichever responses first is redirected to the client. This
> > > might well be the same every time. This might be the one closest to the
> > > mirror302 and not to the client. Is that correct or did I miss a
> weighing
> > > algorithm hidden in there?
> > >
> > > Good coding,
> > >
> > > On 03/03/17 00:23, "Will Stevens"  wrote:
> > >
> > > Hey All,
> > > Please review this repo and tell me what you think:
> > > 

[GitHub] cloudstack pull request #1974: CLOUDSTACK-9795: moved logrotate from cron.da...

[dmabry]

Github user dmabry closed the pull request at:

https://github.com/apache/cloudstack/pull/1974


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1974: CLOUDSTACK-9795: moved logrotate from cron.daily to ...

[dmabry]

Github user dmabry commented on the issue:

https://github.com/apache/cloudstack/pull/1974
  
Thanks @rhtyd!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Modern template hosting

[Rafael Weingärtner]

I think I understood you, but I did not understand you.

Let me see if I can create an example to illustrate how we could do using
the HTTP 300 code.

The HTTP 300 code indicates clients that he/she/it has multiple choices to
access the requested resource. If the request is a HEAD, this is the only
thing that the server returns. If the request if a GET, then the server
returns the choices for the client. The response type is defined on the
server based on the Content-Type, user agent and others.

Let’s say the server receives a GET request and informs the content-type as
JSON (application/JSON). Your application could get the mirrors list, do
the appending process for the requested resource path, and then return a
JSON array with the possible mirror the client can use. Then, the client
gets this list and does the processing required to select a mirror.

Now let’s say that a human access the link using a browser. Then, the
server should receive something like (text/plain) as the content-type; I
think depending on the browser this field may differ a little (it would
require some checking). Anyways, the server detects that it is a “human”
requesting the resource, then we could serve an HTML page with a Javascript
that uses the mirror list. This Javascript could do some checking and
choose the best mirror for that specific.

Why do I think this approach is interesting?
The application you developed would be used only to retrieve valid and
trusted sources of system VM images (a trusted repository for mirrors that
we as a community vouch for).  Also, we shift the decision process
regarding mirrors from the server to the client. Then, it is up to clients
to select mirrors, and not up to us (or some of our implementations).

Did this help?
BTW: I am not saying I am against the way you proposed, which would work
fine. It is merely a suggestion using a different perspective.

On Fri, Mar 3, 2017 at 10:23 AM, Will Stevens 
wrote:

> @rafael: in general, I don't think the client should ever be making a
> choice about a mirror. We have to assume we are working with a scripted
> application and anything hard coded in that implementation on the client
> side is a risk.
>
> I may not be understanding the use of the 300 approach, so I will research
> it to see if I can make it fit.
>
> On Mar 3, 2017 9:59 AM, "Rafael Weingärtner" 
> wrote:
>
> Will great job.
>
> I had the same doubt as Daan.
>
> Have you considered using HTTP 300 working mode? Then, we could let the
> client decide which mirror is the best(closest?). Section “10.3.1” of [1]
> talks about it; this would put the pressure on deciding upon a mirror on
> the client side, but I think it would be fair.
>
> [1] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
>
> PS: You really like very short variables names!
>
> On Fri, Mar 3, 2017 at 1:24 AM, Daan Hoogland  >
> wrote:
>
> > Nice little thing Will,
> >
> > One question: if I read the code correctly it ‘go’es and tries all
> mirrors
> > at once and whichever responses first is redirected to the client. This
> > might well be the same every time. This might be the one closest to the
> > mirror302 and not to the client. Is that correct or did I miss a weighing
> > algorithm hidden in there?
> >
> > Good coding,
> >
> > On 03/03/17 00:23, "Will Stevens"  wrote:
> >
> > Hey All,
> > Please review this repo and tell me what you think:
> > https://github.com/swill/mirror302
> >
> > Let me know if you have questions or you would like me to make
> changes
> > before I bring this topic up with ASF Infra and open a ticket to get
> > this
> > implemented.
> >
> > Cheers,
> >
> > *Will Stevens*
> >
> >
> >
> > On Wed, Mar 1, 2017 at 11:37 AM, Will Stevens  >
> > wrote:
> >
> > > I am building a short term solution right now.  I hopefully will
> > have the
> > > building blocks in place by the weekend so we can start working
> with
> > Infra
> > > to get it in place.
> > >
> > > We will have a web server which we can point something like
> > > downloads.cloudstack.org (or whatever url) at.  Then we will have
> > an ASF
> > > repo which tracks a mirror list and exposes a static site to be
> able
> > to
> > > query the mirror list.
> > >
> > > The web server will receive a request and will do a 302 redirect to
> > the
> > > appropriate resource on one of the mirrors.
> > >
> > > This gets us started.  Legacy environments will have to do a DB
> > change to
> > > change from 'download.cloud.com' to 'downloads.cloudstack.org' (or
> > > whatever), but otherwise it should be pretty simple.
> > >
> > > We can improve how we deliver templates going forward, but this
> > obviously
> > > requires some discussion still.  I will try to buy us some time
> with
> > an
> > > implementation which 

Re: Some Jira tickets clean up

[Rafael Weingärtner]

Tickets closed.

Depending on the tickets, some I marked as closed, others as resolved. I
also configured the status of some to invalid, others to incomplete, others
as fixed.

On Wed, Mar 1, 2017 at 5:07 PM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> Volks,
>
> It has been almost a week. I am assuming that no one knows about these
> tickets.
>
> In the last years, our workflow has been improved greatly; most of the
> PRs I see have Jira ticket associated and everything seems to be getting
> better by the time. However, we have some (a lot?) Jira tickets that seem
> to be left behind. I think a cleanup from time to time is a good thing.
>
>
> In 48 ours I will close all of the tickets mentioned here. If I close
> something that should not have been closed, please pardon me ;).
>
>
>
> After this, I will proceed with a new batch of tickets. Hopefully, in some
> iterations of this process, we can get rid of these forgotten tickets.
>
> On Thu, Feb 23, 2017 at 3:16 PM, Rafael Weingärtner <
> rafaelweingart...@gmail.com> wrote:
>
>> Thanks ;)
>>
>> On Thu, Feb 23, 2017 at 3:14 PM, Tutkowski, Mike <
>> mike.tutkow...@netapp.com> wrote:
>>
>>> Thanks, Rafael. I uploaded an ISO and walked through the necessary parts
>>> of the wizard. I agree with you that it has been fixed. I commented on the
>>> ticket, then resolved and closed it just now.
>>>
>>> On 2/23/17, 12:46 PM, "Rafael Weingärtner" 
>>> wrote:
>>>
>>> Thanks for taking time to look at this Mike.
>>> The ticket (CLOUDSTACK-4164
>>> ) I managed
>>> to
>>> check, and I believe it can be closed.
>>>
>>> On Thu, Feb 23, 2017 at 2:39 PM, Tutkowski, Mike <
>>> mike.tutkow...@netapp.com>
>>> wrote:
>>>
>>> > Thanks for compiling this list, Rafael!
>>> >
>>> > I see I was the reporter for this issue:
>>> >
>>> > https://issues.apache.org/jira/browse/CLOUDSTACK-4164
>>> >
>>> > I plan to take a look at it and see what the current state is.
>>> >
>>> > On 2/23/17, 10:26 AM, "Rafael Weingärtner" <
>>> rafaelweingart...@gmail.com>
>>> > wrote:
>>> >
>>> > Hi folks,
>>> > I have been reviewing old Jira tickets and we have a bunch
>>> that can be
>>> > closed. The following are the ones I have gone through. Before
>>> I close
>>> > any
>>> > ticket I would like to check them with you, especially the
>>> ones from a
>>> > category called “Seems to be fixed has to be checked”. I will
>>> be
>>> > waiting
>>> > for your feedback before I proceed to close any ticket.
>>> >
>>> > - Have a title that does say much and the ticket does not have
>>> a
>>> > detailed
>>> > description
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-2225
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-917
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-947
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-1259
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-1441
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-1576
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-2705
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-5793
>>> >
>>> >
>>> > - Already solved
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-2532
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-6735
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-6837
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-7161
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-8202
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-8781
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-8817
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-9065
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-1069
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-1460
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-1702
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-1870
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-2075
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-2213
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-2439
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-6009
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-5219
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-4847
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-4778
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-4238
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-4164
>>> > -- https://issues.apache.org/jira/browse/CLOUDSTACK-4071
>>> >  

Re: Modern template hosting

[Will Stevens]

@rafael: in general, I don't think the client should ever be making a
choice about a mirror. We have to assume we are working with a scripted
application and anything hard coded in that implementation on the client
side is a risk.

I may not be understanding the use of the 300 approach, so I will research
it to see if I can make it fit.

On Mar 3, 2017 9:59 AM, "Rafael Weingärtner" 
wrote:

Will great job.

I had the same doubt as Daan.

Have you considered using HTTP 300 working mode? Then, we could let the
client decide which mirror is the best(closest?). Section “10.3.1” of [1]
talks about it; this would put the pressure on deciding upon a mirror on
the client side, but I think it would be fair.

[1] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

PS: You really like very short variables names!

On Fri, Mar 3, 2017 at 1:24 AM, Daan Hoogland 
wrote:

> Nice little thing Will,
>
> One question: if I read the code correctly it ‘go’es and tries all mirrors
> at once and whichever responses first is redirected to the client. This
> might well be the same every time. This might be the one closest to the
> mirror302 and not to the client. Is that correct or did I miss a weighing
> algorithm hidden in there?
>
> Good coding,
>
> On 03/03/17 00:23, "Will Stevens"  wrote:
>
> Hey All,
> Please review this repo and tell me what you think:
> https://github.com/swill/mirror302
>
> Let me know if you have questions or you would like me to make changes
> before I bring this topic up with ASF Infra and open a ticket to get
> this
> implemented.
>
> Cheers,
>
> *Will Stevens*
>
>
>
> On Wed, Mar 1, 2017 at 11:37 AM, Will Stevens 
> wrote:
>
> > I am building a short term solution right now.  I hopefully will
> have the
> > building blocks in place by the weekend so we can start working with
> Infra
> > to get it in place.
> >
> > We will have a web server which we can point something like
> > downloads.cloudstack.org (or whatever url) at.  Then we will have
> an ASF
> > repo which tracks a mirror list and exposes a static site to be able
> to
> > query the mirror list.
> >
> > The web server will receive a request and will do a 302 redirect to
> the
> > appropriate resource on one of the mirrors.
> >
> > This gets us started.  Legacy environments will have to do a DB
> change to
> > change from 'download.cloud.com' to 'downloads.cloudstack.org' (or
> > whatever), but otherwise it should be pretty simple.
> >
> > We can improve how we deliver templates going forward, but this
> obviously
> > requires some discussion still.  I will try to buy us some time with
> an
> > implementation which solves our problems today...
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > 
> >
> > On Wed, Mar 1, 2017 at 7:52 AM, Abhinandan Prateek <
> > abhinandan.prat...@shapeblue.com> wrote:
> >
> >>
> >> Initial seeding is a manual step and that is not going to change in
> near
> >> future. A handy list of official places from where these templates
> can be
> >> downloaded won’t harm this part of the installation. Just allow
> anyone
> >> credible who follows democratic process to publish their templates
> with
> >> apache’s blessings.
> >>
> >>
> >> Coming to urls hard coded in db, the popular suggestion is to point
> them
> >> to mirrors. This looks good but this is going to take some effort
> that has
> >> not materialised yet. Once this change is made it will be going in
> some
> >> future release like 4.11; the problem still remains for the
> releases that
> >> are already out there. In case access to download.cloud.com is
> dropped
> >> at some point we will have to document the procedure such that
> people know
> >> how to make these stable releases work. That is where the procedure
> >> documented here https://shankerbalan.net/blog/
> >> seed-cloudstack-templates-offline/ should be officially adapted
> with a
> >> marketplace to pick up templates of choosing. Maybe we can have a
> pre-setup
> >> script to make choices instead of written wiki. Note that this will
> be
> >> anyway needed by the older releases.
> >>
> >>
> >>
> >>
> >> On 01/03/17, 4:24 PM, "Paul Angus" 
> wrote:
> >>
> >> >-1
> >> >
> >> >We are trying to get MORE people to use CloudStack.  And raising
> the
> >> barrier to entry does nothing to help that.
> >> >
> >> >
> >> >Kind regards,
> >> >
> >> >Paul Angus
> >> >
> >> >paul.an...@shapeblue.com
> >> >www.shapeblue.com
> >> >53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> >> >@shapeblue
> >> >
> >> >
> >> >
> >> >
> >> 

Re: Modern template hosting

[Will Stevens]

I can look at the 300 approach. Thanks for the suggestion.

Haha. Ya, I meant to go back and expand the variables in the inline
function. Old habit when using inline functions so it is easier to see them
when scanning the code. I need to stop doing that. :)

On Mar 3, 2017 9:59 AM, "Rafael Weingärtner" 
wrote:

Will great job.

I had the same doubt as Daan.

Have you considered using HTTP 300 working mode? Then, we could let the
client decide which mirror is the best(closest?). Section “10.3.1” of [1]
talks about it; this would put the pressure on deciding upon a mirror on
the client side, but I think it would be fair.

[1] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

PS: You really like very short variables names!

On Fri, Mar 3, 2017 at 1:24 AM, Daan Hoogland 
wrote:

> Nice little thing Will,
>
> One question: if I read the code correctly it ‘go’es and tries all mirrors
> at once and whichever responses first is redirected to the client. This
> might well be the same every time. This might be the one closest to the
> mirror302 and not to the client. Is that correct or did I miss a weighing
> algorithm hidden in there?
>
> Good coding,
>
> On 03/03/17 00:23, "Will Stevens"  wrote:
>
> Hey All,
> Please review this repo and tell me what you think:
> https://github.com/swill/mirror302
>
> Let me know if you have questions or you would like me to make changes
> before I bring this topic up with ASF Infra and open a ticket to get
> this
> implemented.
>
> Cheers,
>
> *Will Stevens*
>
>
>
> On Wed, Mar 1, 2017 at 11:37 AM, Will Stevens 
> wrote:
>
> > I am building a short term solution right now.  I hopefully will
> have the
> > building blocks in place by the weekend so we can start working with
> Infra
> > to get it in place.
> >
> > We will have a web server which we can point something like
> > downloads.cloudstack.org (or whatever url) at.  Then we will have
> an ASF
> > repo which tracks a mirror list and exposes a static site to be able
> to
> > query the mirror list.
> >
> > The web server will receive a request and will do a 302 redirect to
> the
> > appropriate resource on one of the mirrors.
> >
> > This gets us started.  Legacy environments will have to do a DB
> change to
> > change from 'download.cloud.com' to 'downloads.cloudstack.org' (or
> > whatever), but otherwise it should be pretty simple.
> >
> > We can improve how we deliver templates going forward, but this
> obviously
> > requires some discussion still.  I will try to buy us some time with
> an
> > implementation which solves our problems today...
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > 
> >
> > On Wed, Mar 1, 2017 at 7:52 AM, Abhinandan Prateek <
> > abhinandan.prat...@shapeblue.com> wrote:
> >
> >>
> >> Initial seeding is a manual step and that is not going to change in
> near
> >> future. A handy list of official places from where these templates
> can be
> >> downloaded won’t harm this part of the installation. Just allow
> anyone
> >> credible who follows democratic process to publish their templates
> with
> >> apache’s blessings.
> >>
> >>
> >> Coming to urls hard coded in db, the popular suggestion is to point
> them
> >> to mirrors. This looks good but this is going to take some effort
> that has
> >> not materialised yet. Once this change is made it will be going in
> some
> >> future release like 4.11; the problem still remains for the
> releases that
> >> are already out there. In case access to download.cloud.com is
> dropped
> >> at some point we will have to document the procedure such that
> people know
> >> how to make these stable releases work. That is where the procedure
> >> documented here https://shankerbalan.net/blog/
> >> seed-cloudstack-templates-offline/ should be officially adapted
> with a
> >> marketplace to pick up templates of choosing. Maybe we can have a
> pre-setup
> >> script to make choices instead of written wiki. Note that this will
> be
> >> anyway needed by the older releases.
> >>
> >>
> >>
> >>
> >> On 01/03/17, 4:24 PM, "Paul Angus" 
> wrote:
> >>
> >> >-1
> >> >
> >> >We are trying to get MORE people to use CloudStack.  And raising
> the
> >> barrier to entry does nothing to help that.
> >> >
> >> >
> >> >Kind regards,
> >> >
> >> >Paul Angus
> >> >
> >> >paul.an...@shapeblue.com
> >> >www.shapeblue.com
> >> >53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> >> >@shapeblue
> >> >
> >> >
> >> >
> >> >
> >> >-Original Message-
> >> >From: Abhinandan Prateek 

Re: Modern template hosting

[Rafael Weingärtner]

Will great job.

I had the same doubt as Daan.

Have you considered using HTTP 300 working mode? Then, we could let the
client decide which mirror is the best(closest?). Section “10.3.1” of [1]
talks about it; this would put the pressure on deciding upon a mirror on
the client side, but I think it would be fair.

[1] https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

PS: You really like very short variables names!

On Fri, Mar 3, 2017 at 1:24 AM, Daan Hoogland 
wrote:

> Nice little thing Will,
>
> One question: if I read the code correctly it ‘go’es and tries all mirrors
> at once and whichever responses first is redirected to the client. This
> might well be the same every time. This might be the one closest to the
> mirror302 and not to the client. Is that correct or did I miss a weighing
> algorithm hidden in there?
>
> Good coding,
>
> On 03/03/17 00:23, "Will Stevens"  wrote:
>
> Hey All,
> Please review this repo and tell me what you think:
> https://github.com/swill/mirror302
>
> Let me know if you have questions or you would like me to make changes
> before I bring this topic up with ASF Infra and open a ticket to get
> this
> implemented.
>
> Cheers,
>
> *Will Stevens*
>
>
>
> On Wed, Mar 1, 2017 at 11:37 AM, Will Stevens 
> wrote:
>
> > I am building a short term solution right now.  I hopefully will
> have the
> > building blocks in place by the weekend so we can start working with
> Infra
> > to get it in place.
> >
> > We will have a web server which we can point something like
> > downloads.cloudstack.org (or whatever url) at.  Then we will have
> an ASF
> > repo which tracks a mirror list and exposes a static site to be able
> to
> > query the mirror list.
> >
> > The web server will receive a request and will do a 302 redirect to
> the
> > appropriate resource on one of the mirrors.
> >
> > This gets us started.  Legacy environments will have to do a DB
> change to
> > change from 'download.cloud.com' to 'downloads.cloudstack.org' (or
> > whatever), but otherwise it should be pretty simple.
> >
> > We can improve how we deliver templates going forward, but this
> obviously
> > requires some discussion still.  I will try to buy us some time with
> an
> > implementation which solves our problems today...
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > 
> >
> > On Wed, Mar 1, 2017 at 7:52 AM, Abhinandan Prateek <
> > abhinandan.prat...@shapeblue.com> wrote:
> >
> >>
> >> Initial seeding is a manual step and that is not going to change in
> near
> >> future. A handy list of official places from where these templates
> can be
> >> downloaded won’t harm this part of the installation. Just allow
> anyone
> >> credible who follows democratic process to publish their templates
> with
> >> apache’s blessings.
> >>
> >>
> >> Coming to urls hard coded in db, the popular suggestion is to point
> them
> >> to mirrors. This looks good but this is going to take some effort
> that has
> >> not materialised yet. Once this change is made it will be going in
> some
> >> future release like 4.11; the problem still remains for the
> releases that
> >> are already out there. In case access to download.cloud.com is
> dropped
> >> at some point we will have to document the procedure such that
> people know
> >> how to make these stable releases work. That is where the procedure
> >> documented here https://shankerbalan.net/blog/
> >> seed-cloudstack-templates-offline/ should be officially adapted
> with a
> >> marketplace to pick up templates of choosing. Maybe we can have a
> pre-setup
> >> script to make choices instead of written wiki. Note that this will
> be
> >> anyway needed by the older releases.
> >>
> >>
> >>
> >>
> >> On 01/03/17, 4:24 PM, "Paul Angus" 
> wrote:
> >>
> >> >-1
> >> >
> >> >We are trying to get MORE people to use CloudStack.  And raising
> the
> >> barrier to entry does nothing to help that.
> >> >
> >> >
> >> >Kind regards,
> >> >
> >> >Paul Angus
> >> >
> >> >paul.an...@shapeblue.com
> >> >www.shapeblue.com
> >> >53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> >> >@shapeblue
> >> >
> >> >
> >> >
> >> >
> >> >-Original Message-
> >> >From: Abhinandan Prateek [mailto:abhinandan.prat...@shapeblue.com]
> >> >Sent: 01 March 2017 09:39
> >> >To: dev@cloudstack.apache.org
> >> >Subject: Re: Modern template hosting
> >> >
> >> >+1
> >> >
> >> >After seeing this discussion running in circles several times, I
> think
> >> we should at least get started with the simplest option.
> >> >
> >> >

Re: Modern template hosting

[Will Stevens]

You understand the logic correctly.

Yes, it will favor mirrors which are closest to the hosting of mirror302.
However, if there is a problem with a mirror being overloaded, another
mirror will win the race and serve the resource.

I originally started writing it to pick the closest mirrors geographically
and then verify with head that the selected mirror has the requested
resource.
With this approach I was getting the feeling that I was over complicating
the problem. How do I verify if the closest mirror is having problems.
Let's say it is over loaded or has some other problem and takes twice as
long to deliver the resource as a different mirror. How would I be able to
verify this and be sure the resource can be served in a timely fashion?

The geo way is very deterministic about which mirror is chosen. That is
good for most cases, but if a mirror is having a hard time but is still
responding to head, there does not seem to be a good way for the impacted
client to delivered a different mirror.

I released the code with this simplified implementation for a few reasons.

1) it is easier to understand and is more likely to handle the case that a
mirror is struggling and not direct clients to it.
2) verify with the community that this approach to solving the problem is
acceptable before spending a lot of time to 'perfect' it.
3) get feedback on the issues around geo selection before trying to do a
final implementation.
4) have something to show in order to open a ticket with ASF infra to start
working through the process of getting this in place.

One other note around where the mirror list is hosted. Maybe we can serve
the mirror list from cloudstack-www. We already have automatic publishing
of the static resources on commit to the ASF. Is there a reason we need
anything more complicated than that?

Anyway, would love feedback all around. I am happy to adapt this as we see
fit, but want to make sure we are generally happy with the approach before
I invest a bunch of time into it.

Thanks for the review. :)

On Mar 3, 2017 1:24 AM, "Daan Hoogland"  wrote:

> Nice little thing Will,
>
> One question: if I read the code correctly it ‘go’es and tries all mirrors
> at once and whichever responses first is redirected to the client. This
> might well be the same every time. This might be the one closest to the
> mirror302 and not to the client. Is that correct or did I miss a weighing
> algorithm hidden in there?
>
> Good coding,
>
> On 03/03/17 00:23, "Will Stevens"  wrote:
>
> Hey All,
> Please review this repo and tell me what you think:
> https://github.com/swill/mirror302
>
> Let me know if you have questions or you would like me to make changes
> before I bring this topic up with ASF Infra and open a ticket to get
> this
> implemented.
>
> Cheers,
>
> *Will Stevens*
>
>
>
> On Wed, Mar 1, 2017 at 11:37 AM, Will Stevens 
> wrote:
>
> > I am building a short term solution right now.  I hopefully will
> have the
> > building blocks in place by the weekend so we can start working with
> Infra
> > to get it in place.
> >
> > We will have a web server which we can point something like
> > downloads.cloudstack.org (or whatever url) at.  Then we will have
> an ASF
> > repo which tracks a mirror list and exposes a static site to be able
> to
> > query the mirror list.
> >
> > The web server will receive a request and will do a 302 redirect to
> the
> > appropriate resource on one of the mirrors.
> >
> > This gets us started.  Legacy environments will have to do a DB
> change to
> > change from 'download.cloud.com' to 'downloads.cloudstack.org' (or
> > whatever), but otherwise it should be pretty simple.
> >
> > We can improve how we deliver templates going forward, but this
> obviously
> > requires some discussion still.  I will try to buy us some time with
> an
> > implementation which solves our problems today...
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > 
> >
> > On Wed, Mar 1, 2017 at 7:52 AM, Abhinandan Prateek <
> > abhinandan.prat...@shapeblue.com> wrote:
> >
> >>
> >> Initial seeding is a manual step and that is not going to change in
> near
> >> future. A handy list of official places from where these templates
> can be
> >> downloaded won’t harm this part of the installation. Just allow
> anyone
> >> credible who follows democratic process to publish their templates
> with
> >> apache’s blessings.
> >>
> >>
> >> Coming to urls hard coded in db, the popular suggestion is to point
> them
> >> to mirrors. This looks good but this is going to take some effort
> that has
> >> not materialised yet. Once this change is made it will be going in
> some
> >> future release like 4.11; the problem still remains for the
> 

[GitHub] cloudstack issue #1776: CLOUDSTACK-9603: 'concurrent.snapshots.threshold.per...

[sateesh-chodapuneedi]

Github user sateesh-chodapuneedi commented on the issue:

https://github.com/apache/cloudstack/pull/1776
  
@borisstoyanov Can you please trigger tests for this PR? Tests could be 
over any hypervisor/simulator as the fix is pretty generic.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #892: CLOUDSTACK-8910: The reserved_capacity field increase...

[SudharmaJain]

Github user SudharmaJain commented on the issue:

https://github.com/apache/cloudstack/pull/892
  
@alexandrelimassantana As suggested I have moved the code snippet to a 
method and also added unit tests. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1862: CLOUDSTACK-9704 Remove dependency on VmwareContext o...

[sateesh-chodapuneedi]

Github user sateesh-chodapuneedi commented on the issue:

https://github.com/apache/cloudstack/pull/1862
  
Thanks @borisstoyanov 

The 3 test failures listed in results doesn't seem related to the code 
changes in PR. 

```
2017-03-02 17:45:11,699 - CRITICAL - EXCEPTION: 
test_02_list_snapshots_with_removed_data_store: ['Traceback (most recent call 
last):\n', '  File "/usr/lib64/python2.7/unittest/case.py", line 369, in run\n  
  testMethod()\n', '  File 
"/usr/lib/python2.7/site-packages/marvin/lib/decoratorGenerators.py", line 30, 
in test_wrapper\nreturn test(self, *args, **kwargs)\n', '  File 
"/marvin/tests/smoke/test_snapshots.py", line 363, in 
test_02_list_snapshots_with_removed_data_store\nid=snapshot.id\n', '  File 
"/usr/lib/python2.7/site-packages/marvin/lib/base.py", line 1125, in list\n
return(apiclient.listSnapshots(cmd))\n', '  File 
"/usr/lib/python2.7/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py", 
line 1161, in listSnapshots\nresponse = 
self.connection.marvinRequest(command, response_type=response, 
method=method)\n', '  File 
"/usr/lib/python2.7/site-packages/marvin/cloudstackConnection.py", line 379, in 
marvinRequest\nraise e\n', 'CloudstackAPIException: Exe
 cute cmd: listsnapshots failed, due to: errorCode: 530, errorText:Unable to 
locate datastore with id 5\n']

2017-03-02 17:45:16,772 - CRITICAL - EXCEPTION: 
test_02_list_snapshots_with_removed_data_store: ['Traceback (most recent call 
last):\n', '  File "/usr/lib64/python2.7/unittest/case.py", line 398, in run\n  
  self.tearDown()\n', '  File "/marvin/tests/smoke/test_snapshots.py", line 
174, in tearDown\nraise Exception("Warning: Exception during cleanup : %s" 
% e)\n', "Exception: Warning: Exception during cleanup : Job failed: 
{jobprocstatus : 0, created : u'2017-03-02T17:45:13+', jobresult : 
{errorcode : 431, errortext : u'Unable to find a virtual machine with specified 
vmId'}, cmd : 
u'org.apache.cloudstack.api.command.admin.vm.DestroyVMCmdByAdmin', userid : 
u'87233b2e-ff35-11e6-be08-06674c01070f', jobstatus : 2, jobid : 
u'43f0ee45-e763-45bc-b777-795c7eaeb26a', jobresultcode : 530, jobinstanceid : 
u'3be27ffe-1884-493b-9566-e616cf2d85d0', jobresulttype : u'object', 
jobinstancetype : u'VirtualMachine', accountid : 
u'872334ae-ff35-11e6-be08-06674c01070f'}\n"]

2017-03-02 17:45:41,985 - CRITICAL - EXCEPTION: 
test_02_list_snapshots_with_removed_data_store: ['Traceback (most recent call 
last):\n', '  File "/usr/lib/python2.7/site-packages/nose/suite.py", line 228, 
in run\nself.tearDown()\n', '  File 
"/usr/lib/python2.7/site-packages/nose/suite.py", line 351, in tearDown\n
self.teardownContext(ancestor)\n', '  File 
"/usr/lib/python2.7/site-packages/nose/suite.py", line 367, in 
teardownContext\ntry_run(context, names)\n', '  File 
"/usr/lib/python2.7/site-packages/nose/util.py", line 471, in try_run\n
return func()\n', '  File "/marvin/tests/smoke/test_snapshots.py", line 160, in 
tearDownClass\nraise Exception("Warning: Exception during cleanup : %s" % 
e)\n', "Exception: Warning: Exception during cleanup : Job failed: 
{jobprocstatus : 0, created : u'2017-03-02T17:45:39+', jobresult : 
{errorcode : 530, errortext : u'Failed to delete template'}, cmd : 
u'org.apache.cloudstack.api.command.user.template.DeleteTemplateCmd', u
 serid : u'87233b2e-ff35-11e6-be08-06674c01070f', jobstatus : 2, jobid : 
u'df91fb7f-b9fd-4b9f-8d07-654821c42d69', jobresultcode : 530, jobinstanceid : 
u'5fc32822-46b8-43e9-ac33-8ad5bb00e775', jobresulttype : u'object', 
jobinstancetype : u'Template', accountid : 
u'872334ae-ff35-11e6-be08-06674c01070f'}\n"]

```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1983: [4.10/blocker] Upgrade4920to41000: Migrate systemvmt...

[blueorangutan]

Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1983
  
@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been 
kicked to run smoke tests


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1983: [4.10/blocker] Upgrade4920to41000: Migrate systemvmt...

[rhtyd]

Github user rhtyd commented on the issue:

https://github.com/apache/cloudstack/pull/1983
  
@blueorangutan test


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1877: CLOUDSTACK-9716: Change of network to rvr for bareme...

[SudharmaJain]

Github user SudharmaJain commented on the issue:

https://github.com/apache/cloudstack/pull/1877
  
@rashmidixit Could you add a marvin test case? It looks with your changes, 
master VR Redundant state will be stuck in unknown state. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1987: CLOUDSTACK-9814 : Unable to edit a Sub domain...

[yvsubhash]

Github user yvsubhash commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1987#discussion_r104140435
  
--- Diff: server/src/org/apache/cloudstack/region/RegionManagerImpl.java ---
@@ -229,7 +232,7 @@ public boolean deleteUser(DeleteUserCmd cmd) {
  */
 @Override
 public Domain updateDomain(UpdateDomainCmd cmd) {
-return _domainMgr.updateDomain(cmd);
+return _managementService.updateDomain(cmd);
--- End diff --

Instead of changing the method that is getting called, the missing one in 
search query can be added here and it seems the updateDomain method in 
managementserviceimpl is not in use. It can be removed


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1987: CLOUDSTACK-9814 : Unable to edit a Sub domain...

[niteshsarda]

GitHub user niteshsarda opened a pull request:

https://github.com/apache/cloudstack/pull/1987

CLOUDSTACK-9814 : Unable to edit a Sub domain, which has the same name in 
different domains

ISSUE

Unable to edit a Sub domain, which has the same name in different domains.

TROUBLESHOOTING
==
1. Created two sub-domains with same name. Under two different domains.
ROOT
Domain 1 : Sample
AA01
BB01

Domain 2 : Test
AA01

2. Try to edit resource - Click on edit - edit any parameter.

EXPECTED BEHAVIOR
==
It should allow to edit the required parameter and update the value.

ACTUAL BEHAVIOR
==
It throws an error "Failed to update specified domain id with name 
'AA01' since it already exists in system. 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Accelerite/cloudstack SubDomainIssue

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1987.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1987


commit 2d29b4fc48081efe683457c7f4597c643f3fa480
Author: Nitesh Sarda 
Date:   2017-03-03T11:24:48Z

CLOUDSTACK-9814 : Unable to edit a Sub domain, which has the same name in 
different domains




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1986: CLOUDSTACK-9814 : Unable to edit a Sub domain...

[niteshsarda]

Github user niteshsarda closed the pull request at:

https://github.com/apache/cloudstack/pull/1986


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1986: CLOUDSTACK-9814 : Unable to edit a Sub domain...

[niteshsarda]

GitHub user niteshsarda opened a pull request:

https://github.com/apache/cloudstack/pull/1986

CLOUDSTACK-9814 : Unable to edit a Sub domain, which has the same name in 
different domains

ISSUE

Unable to edit a Sub domain, which has the same name in different domains.

TROUBLESHOOTING
==
1. Created two sub-domains with same name. Under two different domains.
ROOT
Domain 1 : India@b582a4cb-031e-49e7-afcf-4281fbf231b4
AA01
BB01

Domain 2 : Test
AA01

2. Try to edit resource - Click on edit - edit any parameter.

EXPECTED BEHAVIOR
==
It should allow to edit the required parameter and update the value.

ACTUAL BEHAVIOR
==
It throws an error "Failed to update specified domain id with name 
'AA01' since it already exists in system. 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Accelerite/cloudstack CS-50514

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1986.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1986


commit 2f75b51f42c9400a7be172e405a4ff13bf4c2722
Author: Nitesh Sarda 
Date:   2017-02-27T17:54:20Z

CLOUDSTACK-9779 : Releasing secondary guest IP fails with error VM nic Ip 
x.x.x.x is mapped to load balancing rule

commit ff97817b907c25aff8ccf9bfd36a1d12966b6d9e
Author: Nitesh Sarda 
Date:   2017-03-03T10:47:58Z

CLOUDSTACK-9814 : Unable to edit a Sub domain, which has the same name in 
different domains




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1984: CLOUDSTACK-9812:Update "updatePortForwardingRule" pi...

[mrunalinikankariya]

Github user mrunalinikankariya commented on the issue:

https://github.com/apache/cloudstack/pull/1984
  
Can see two cmmits in same card. so 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1984: CLOUDSTACK-9812:Update "updatePortForwardingR...

[mrunalinikankariya]

Github user mrunalinikankariya closed the pull request at:

https://github.com/apache/cloudstack/pull/1984


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1985: CLOUDSTACK-9812:Update "updatePortForwardingR...

[mrunalinikankariya]

GitHub user mrunalinikankariya opened a pull request:

https://github.com/apache/cloudstack/pull/1985

CLOUDSTACK-9812:Update "updatePortForwardingRule" api to include additional 
parameter to update the end port in case of port range

Configure a PF rule Private port : Start port ; 20 ENd POrt 25 || Public 
Port : Start port 20 ; ENd Port : 25.
Trigger UpdatePortForwardingRule api
ApI fails with following error : " Unable to update the private port of 
port forwarding rule as the rule has port range "

Solution-
Port range gets modified

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Accelerite/cloudstack cloudstack-9812

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1985.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1985


commit c38e01b46e13411638b8a84a02fbad91b31ecbd4
Author: Mrunalini Kankariya 
Date:   2017-03-03T08:24:10Z

CLOUDSTACK-9812:Update updatePortForwardingRule api to include additional 
parameter to update the end port in case of port range




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: :[VOTE] Apache Cloudstack 4.10.0.0

[Rajani Karuturi]

I will create RC2 on Monday with the fixes mentioned in my
previous mail.

~ Rajani

http://cloudplatform.accelerite.com/

On March 3, 2017 at 2:36 PM, Rohit Yadav
(rohit.ya...@shapeblue.com) wrote:

Thanks Koushik, I did not realize Kishan had sent this already.
Let's get either of the PRs merged and kick a RC2.

Regards.


From: Koushik Das 
Sent: 03 March 2017 14:14:56
To: dev@cloudstack.apache.org
Subject: Re: :[VOTE] Apache Cloudstack 4.10.0.0

Looks like there is already a PR for the same issue
https://github.com/apache/cloudstack/pull/1982 from Kishan.

-Koushik

On 03/03/17, 1:58 PM, "Rohit Yadav" 
wrote:

-1 (binding)

All, I've found an upgrade blocker. Pre 4.6 users are required
to seed 4.6 systemvmtemplate to proceed with the upgrade
otherwise upgrade fails, and from 4.9 upgrade to 4.10 does no
check/enforcement that 4.10 based systemvmtemplate has been
seeded/registered, nor the minimum required systemvmtemplate
version is changed from 4.6.0 to 4.10.0.

After we have merged the strongswan/java8 PR, I had updated the
upgrade docs on how to upgrade the systemvmtemplate here:

http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.10/upgrade/upgrade-4.9.html

Using the above, I've tried to fix these issues here, please
review and merge for RC2:

https://github.com/apache/cloudstack/pull/1983

With above fix,
the aim is that users only seed the 4.10 systemvmtemplate before
upgrade and post-upgrade the upgrade paths fix the entries,
global setting etc.

Regards.


From: Tutkowski, Mike 
Sent: 02 March 2017 22:39:08
To: dev@cloudstack.apache.org
Subject: Re: :[VOTE] Apache Cloudstack 4.10.0.0

I rolled back to my master branch at
da66b06e7d562393da2e4b52206943f8bad49d10 and it works.

It appears something that went into after that commit has broken
this. It looks like this SHA is about two weeks old and that 43
commits have gone into master since it.

On 3/2/17, 7:06 AM, "Tutkowski, Mike"
 wrote:

According to where the code fails, though, it appears to be a
networking problem. If I set a breakpoint before the failure and
change a variable to say that security groups are not being used,
then the VM starts.

I think this is a recently introduced problem because I have
another branch based off of a slightly older version of master
and it works fine here.

> On Mar 2, 2017, at 6:51 AM, Pierre-Luc Dion
 wrote:
>
> Hi Mike,
> Try vm with at least 512MB for memory.
>
>> On Mar 1, 2017 15:01, "Tutkowski, Mike"
 wrote:
>>
>> I see the following exception when trying to deploy a user VM
in a Basic
>> Zone with two XenServer 6.5 hosts in one cluster. My system
VMs have all
>> deployed properly. The user template gets downloaded fine. I
can see the
>> user VM begin to start on a XenServer host, then it goes
away. We then
>> automatically try on the other host. I can see the VM begin
to start there
>> for a moment, then it goes away.
>>
>> I am just deploying the user VM’s template and root disk to
NFS (same
>> place where the template and root disks of my system VMs
are).
>>
>> I am using the built-in XenServer CentOS 5.6 (64 bit)
template with 1
>> vCPU, 500 MHz, and 256 MB memory.
>>
>> WARN [c.c.a.r.v.VirtualRoutingResource]
(DirectAgent-7:ctx-35aded78)
>> (logid:aab9c320) Expected 1 answers while executing
VmDataCommand but
>> received 2
>> WARN [c.c.v.VirtualMachinePowerStateSyncImpl]
(DirectAgentCronJob-14:ctx-27fb1ac3)
>> (logid:2c342f23) VM state was updated but update time is
null?! vm id: 6
>> INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-2c7d2dce)
>> (logid:a56a9a8c) Begin cleanup expired async-jobs
>> INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-2c7d2dce)
>> (logid:a56a9a8c) End cleanup expired async-jobs
>> INFO [c.c.u.AccountManagerImpl]
(AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 removed accounts to cleanup
>> INFO [c.c.u.AccountManagerImpl]
(AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 disabled accounts to cleanup
>> INFO [c.c.u.AccountManagerImpl]
(AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 inactive domains to cleanup
>> INFO [c.c.u.AccountManagerImpl]
(AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 disabled projects to cleanup
>> WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-16:ctx-7c901443)
>> (logid:aab9c320) callHostPlugin failed for cmd:
default_network_rules with
>> args secIps: 0:, vmName: i-2-6-VM, vmID: 6, vmIP:
10.117.40.53, vmMAC:
>> 06:b2:f4:00:00:22, due to There was a failure communicating
with the
>> plugin.
>> WARN [c.c.h.x.r.w.x.CitrixStartCommandWrapper]
>> (DirectAgent-16:ctx-7c901443) (logid:aab9c320) Catch
Exception: class
>> com.cloud.utils.exception.CloudRuntimeException due to
>> 

[GitHub] cloudstack issue #1984: CLOUDSTACK-9812:Update "updatePortForwardingRule" pi...

[yvsubhash]

Github user yvsubhash commented on the issue:

https://github.com/apache/cloudstack/pull/1984
  
@mrunalinikankariya One of the commits is not related to the issue


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1779: CLOUDSTACK-9610: Disabled Host Keeps Being up status...

[priyankparihar]

Github user priyankparihar commented on the issue:

https://github.com/apache/cloudstack/pull/1779
  
Hi @DaanHoogland code is modified. Now we are not hiding. It will be shown 
in Disconnected state. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1771: CLOUDSTACK-9611: Dedicating a Guest VLAN range to Pr...

[ustcweizhou]

Github user ustcweizhou commented on the issue:

https://github.com/apache/cloudstack/pull/1771
  
@nitin-maharana should projectId dependsOn domainId ? If the domain 
changes, the project list should change according to the new domainid.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack-docs issue #19: Updating The Nuage VSP Plugin Documentation for A...

[prashanthvarma]

Github user prashanthvarma commented on the issue:

https://github.com/apache/cloudstack-docs/pull/19
  
@rhtyd @karuturi Can we please merge this ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1984: CLOUDSTACK-9812:Update "updatePortForwardingR...

[mrunalinikankariya]

GitHub user mrunalinikankariya opened a pull request:

https://github.com/apache/cloudstack/pull/1984

CLOUDSTACK-9812:Update "updatePortForwardingRule" pi to include additional 
parameter to update the end port in case of port range

Configure a PF rule Private port : Start port ; 20 ENd POrt 25 || Public 
Port : Start port 20 ; ENd Port : 25.
Trigger UpdatePortForwardingRule api
ApI fails with following error : " Unable to update the private port of 
port forwarding rule as the rule has port range "

Solution-
Port range gets modified

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Accelerite/cloudstack cs-9812

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1984.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1984


commit 4f0c80a331b37b928b4c47ff99d32ecf11388d4e
Author: Mrunalini Kankariya 
Date:   2017-01-18T09:45:29Z

CLOUDSTACK-9742:Simultaneous snapshots for detached volume

commit 74a0f6f26c593234abf30fa1b5efcddca9a23dda
Author: Mrunalini Kankariya 
Date:   2017-03-03T08:24:10Z

CLOUDSTACK-9812:Update updatePortForwardingRule api to include additional 
parameter to update the end port in case of port range




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1894: CLOUDSTACK-9700 Allow user to Register/Copy t...

[priyankparihar]

Github user priyankparihar commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1894#discussion_r104114043
  
--- Diff: 
api/src/org/apache/cloudstack/api/command/user/template/CopyTemplateCmd.java ---
@@ -51,25 +52,46 @@
 @Parameter(name = ApiConstants.DESTINATION_ZONE_ID,
type = CommandType.UUID,
entityType = ZoneResponse.class,
-   required = true,
+   required = false,
description = "ID of the zone the template is being copied 
to.")
-private Long destZoneId;
+protected Long destZoneId;
 
-@Parameter(name = ApiConstants.ID, type = CommandType.UUID, entityType 
= TemplateResponse.class, required = true, description = "Template ID.")
+@Parameter(name = ApiConstants.ID, type = CommandType.UUID,
+entityType = TemplateResponse.class, required = true, 
description = "Template ID.")
 private Long id;
 
 @Parameter(name = ApiConstants.SOURCE_ZONE_ID,
type = CommandType.UUID,
entityType = ZoneResponse.class,
-description = "ID of the zone the template is currently hosted 
on. If not specified and template is cross-zone, then we will sync this 
template to region wide image store.")
+description = "ID of the zone the template is currently hosted 
on. " +
+"If not specified and template is cross-zone, " +
+"then we will sync this template to region wide image 
store.")
--- End diff --

This change is not required. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1894: CLOUDSTACK-9700 Allow user to Register/Copy template...

[priyankparihar]

Github user priyankparihar commented on the issue:

https://github.com/apache/cloudstack/pull/1894
  
@rashmidixit can you run internal CI and post @cloudmonger test results?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: :[VOTE] Apache Cloudstack 4.10.0.0

[Rohit Yadav]

Thanks Koushik, I did not realize Kishan had sent this already. Let's get 
either of the PRs merged and kick a RC2.


Regards.


From: Koushik Das 
Sent: 03 March 2017 14:14:56
To: dev@cloudstack.apache.org
Subject: Re: :[VOTE] Apache Cloudstack 4.10.0.0

Looks like there is already a PR for the same issue 
https://github.com/apache/cloudstack/pull/1982 from Kishan.

-Koushik

On 03/03/17, 1:58 PM, "Rohit Yadav"  wrote:

-1 (binding)


All, I've found an upgrade blocker. Pre 4.6 users are required to seed 4.6 
systemvmtemplate to proceed with the upgrade otherwise upgrade fails, and from 
4.9 upgrade to 4.10 does no check/enforcement that 4.10 based systemvmtemplate 
has been seeded/registered, nor the minimum required systemvmtemplate version 
is changed from 4.6.0 to 4.10.0.


After we have merged the strongswan/java8 PR, I had updated the upgrade 
docs on how to upgrade the systemvmtemplate here:


http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.10/upgrade/upgrade-4.9.html


Using the above, I've tried to fix these issues here, please review and 
merge for RC2:

https://github.com/apache/cloudstack/pull/1983


With above fix, the aim is 
that users only seed the 4.10 systemvmtemplate before upgrade and post-upgrade 
the upgrade paths fix the entries, global setting etc.


Regards.


From: Tutkowski, Mike 
Sent: 02 March 2017 22:39:08
To: dev@cloudstack.apache.org
Subject: Re: :[VOTE] Apache Cloudstack 4.10.0.0

I rolled back to my master branch at 
da66b06e7d562393da2e4b52206943f8bad49d10 and it works.

It appears something that went into after that commit has broken this. It 
looks like this SHA is about two weeks old and that 43 commits have gone into 
master since it.

On 3/2/17, 7:06 AM, "Tutkowski, Mike"  wrote:

According to where the code fails, though, it appears to be a 
networking problem. If I set a breakpoint before the failure and change a 
variable to say that security groups are not being used, then the VM starts.

I think this is a recently introduced problem because I have another 
branch based off of a slightly older version of master and it works fine here.

> On Mar 2, 2017, at 6:51 AM, Pierre-Luc Dion  
wrote:
>
> Hi Mike,
> Try vm with at least 512MB for memory.
>
>> On Mar 1, 2017 15:01, "Tutkowski, Mike"  
wrote:
>>
>> I see the following exception when trying to deploy a user VM in a 
Basic
>> Zone with two XenServer 6.5 hosts in one cluster. My system VMs have 
all
>> deployed properly. The user template gets downloaded fine. I can see 
the
>> user VM begin to start on a XenServer host, then it goes away. We 
then
>> automatically try on the other host. I can see the VM begin to start 
there
>> for a moment, then it goes away.
>>
>> I am just deploying the user VM’s template and root disk to NFS (same
>> place where the template and root disks of my system VMs are).
>>
>> I am using the built-in XenServer CentOS 5.6 (64 bit) template with 1
>> vCPU, 500 MHz, and 256 MB memory.
>>
>> WARN  [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-7:ctx-35aded78)
>> (logid:aab9c320) Expected 1 answers while executing VmDataCommand but
>> received 2
>> WARN  [c.c.v.VirtualMachinePowerStateSyncImpl] 
(DirectAgentCronJob-14:ctx-27fb1ac3)
>> (logid:2c342f23) VM state was updated but update time is null?! vm 
id: 6
>> INFO  [o.a.c.f.j.i.AsyncJobManagerImpl] 
(AsyncJobMgr-Heartbeat-1:ctx-2c7d2dce)
>> (logid:a56a9a8c) Begin cleanup expired async-jobs
>> INFO  [o.a.c.f.j.i.AsyncJobManagerImpl] 
(AsyncJobMgr-Heartbeat-1:ctx-2c7d2dce)
>> (logid:a56a9a8c) End cleanup expired async-jobs
>> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 removed accounts to cleanup
>> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 disabled accounts to cleanup
>> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 inactive domains to cleanup
>> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 disabled projects to cleanup
>> WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-16:ctx-7c901443)
>> (logid:aab9c320) callHostPlugin failed for cmd: 
default_network_rules with
>> args secIps: 0:, vmName: i-2-6-VM, vmID: 6, vmIP: 10.117.40.53, 
vmMAC:
>> 

[GitHub] cloudstack issue #1982: CLOUDSTACK-9807/CLOUDSTACK-9808 - Added upgrade chan...

[rhtyd]

Github user rhtyd commented on the issue:

https://github.com/apache/cloudstack/pull/1982
  
Thanks @kishankavala I did not realize we had this already, I created this 
#1983 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1917: CLOUDSTACK-9756: Configure to ignore the ipassoc fai...

[cloudmonger]

Github user cloudmonger commented on the issue:

https://github.com/apache/cloudstack/pull/1917
  
### ACS CI BVT Run
 **Sumarry:**
 Build Number 417
 Hypervisor xenserver
 NetworkType Advanced
 Passed=105
 Failed=0
 Skipped=7

_Link to logs Folder (search by build_no):_ 
https://www.dropbox.com/sh/yj3wnzbceo9uef2/AAB6u-Iap-xztdm6jHX9SjPja?dl=0


**Failed tests:**

**Skipped tests:**
test_01_test_vm_volume_snapshot
test_vm_nic_adapter_vmxnet3
test_static_role_account_acls
test_11_ss_nfs_version_on_ssvm
test_nested_virtualization_vmware
test_3d_gpu_support
test_deploy_vgpu_enabled_vm

**Passed test suits:**
test_deploy_vm_with_userdata.py
test_affinity_groups_projects.py
test_portable_publicip.py
test_over_provisioning.py
test_global_settings.py
test_scale_vm.py
test_service_offerings.py
test_routers_iptables_default_policy.py
test_loadbalance.py
test_routers.py
test_reset_vm_on_reboot.py
test_deploy_vms_with_varied_deploymentplanners.py
test_network.py
test_router_dns.py
test_non_contigiousvlan.py
test_login.py
test_deploy_vm_iso.py
test_list_ids_parameter.py
test_public_ip_range.py
test_multipleips_per_nic.py
test_regions.py
test_affinity_groups.py
test_network_acl.py
test_pvlan.py
test_volumes.py
test_nic.py
test_deploy_vm_root_resize.py
test_resource_detail.py
test_secondary_storage.py
test_vm_life_cycle.py
test_routers_network_ops.py
test_disk_offerings.py


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1983: [4.10/blocker] Upgrade4920to41000: Migrate systemvmt...

[blueorangutan]

Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1983
  
Packaging result: ✔centos6 ✔centos7 ✔debian. JID-560


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: :[VOTE] Apache Cloudstack 4.10.0.0

[Koushik Das]

Looks like there is already a PR for the same issue 
https://github.com/apache/cloudstack/pull/1982 from Kishan.

-Koushik

On 03/03/17, 1:58 PM, "Rohit Yadav"  wrote:

-1 (binding)


All, I've found an upgrade blocker. Pre 4.6 users are required to seed 4.6 
systemvmtemplate to proceed with the upgrade otherwise upgrade fails, and from 
4.9 upgrade to 4.10 does no check/enforcement that 4.10 based systemvmtemplate 
has been seeded/registered, nor the minimum required systemvmtemplate version 
is changed from 4.6.0 to 4.10.0.


After we have merged the strongswan/java8 PR, I had updated the upgrade 
docs on how to upgrade the systemvmtemplate here:


http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.10/upgrade/upgrade-4.9.html


Using the above, I've tried to fix these issues here, please review and 
merge for RC2:

https://github.com/apache/cloudstack/pull/1983


With above fix, the aim is 
that users only seed the 4.10 systemvmtemplate before upgrade and post-upgrade 
the upgrade paths fix the entries, global setting etc.


Regards.


From: Tutkowski, Mike 
Sent: 02 March 2017 22:39:08
To: dev@cloudstack.apache.org
Subject: Re: :[VOTE] Apache Cloudstack 4.10.0.0

I rolled back to my master branch at 
da66b06e7d562393da2e4b52206943f8bad49d10 and it works.

It appears something that went into after that commit has broken this. It 
looks like this SHA is about two weeks old and that 43 commits have gone into 
master since it.

On 3/2/17, 7:06 AM, "Tutkowski, Mike"  wrote:

According to where the code fails, though, it appears to be a 
networking problem. If I set a breakpoint before the failure and change a 
variable to say that security groups are not being used, then the VM starts.

I think this is a recently introduced problem because I have another 
branch based off of a slightly older version of master and it works fine here.

> On Mar 2, 2017, at 6:51 AM, Pierre-Luc Dion  
wrote:
>
> Hi Mike,
> Try vm with at least 512MB for memory.
>
>> On Mar 1, 2017 15:01, "Tutkowski, Mike"  
wrote:
>>
>> I see the following exception when trying to deploy a user VM in a 
Basic
>> Zone with two XenServer 6.5 hosts in one cluster. My system VMs have 
all
>> deployed properly. The user template gets downloaded fine. I can see 
the
>> user VM begin to start on a XenServer host, then it goes away. We 
then
>> automatically try on the other host. I can see the VM begin to start 
there
>> for a moment, then it goes away.
>>
>> I am just deploying the user VM’s template and root disk to NFS (same
>> place where the template and root disks of my system VMs are).
>>
>> I am using the built-in XenServer CentOS 5.6 (64 bit) template with 1
>> vCPU, 500 MHz, and 256 MB memory.
>>
>> WARN  [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-7:ctx-35aded78)
>> (logid:aab9c320) Expected 1 answers while executing VmDataCommand but
>> received 2
>> WARN  [c.c.v.VirtualMachinePowerStateSyncImpl] 
(DirectAgentCronJob-14:ctx-27fb1ac3)
>> (logid:2c342f23) VM state was updated but update time is null?! vm 
id: 6
>> INFO  [o.a.c.f.j.i.AsyncJobManagerImpl] 
(AsyncJobMgr-Heartbeat-1:ctx-2c7d2dce)
>> (logid:a56a9a8c) Begin cleanup expired async-jobs
>> INFO  [o.a.c.f.j.i.AsyncJobManagerImpl] 
(AsyncJobMgr-Heartbeat-1:ctx-2c7d2dce)
>> (logid:a56a9a8c) End cleanup expired async-jobs
>> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 removed accounts to cleanup
>> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 disabled accounts to cleanup
>> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 inactive domains to cleanup
>> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 disabled projects to cleanup
>> WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-16:ctx-7c901443)
>> (logid:aab9c320) callHostPlugin failed for cmd: 
default_network_rules with
>> args secIps: 0:, vmName: i-2-6-VM, vmID: 6, vmIP: 10.117.40.53, 
vmMAC:
>> 06:b2:f4:00:00:22,  due to There was a failure communicating with the
>> plugin.
>> WARN  [c.c.h.x.r.w.x.CitrixStartCommandWrapper]
>> (DirectAgent-16:ctx-7c901443) (logid:aab9c320) Catch Exception: class
>> 

[GitHub] cloudstack issue #1813: CLOUDSTACK-9604: Root disk resize support for VMware...

[blueorangutan]

Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1813
  
@rhtyd a Trillian-Jenkins test job (centos7 mgmt + vmware-55u3) has been 
kicked to run smoke tests


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1813: CLOUDSTACK-9604: Root disk resize support for VMware...

[rhtyd]

Github user rhtyd commented on the issue:

https://github.com/apache/cloudstack/pull/1813
  
@blueorangutan test centos7 vmware-55u3


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1908: CLOUDSTACK-9317: Fixed disable static nat on leaving...

[ProjectMoon]

Github user ProjectMoon commented on the issue:

https://github.com/apache/cloudstack/pull/1908
  
Hi, that's great news.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1974: CLOUDSTACK-9795: moved logrotate from cron.daily to ...

[rhtyd]

Github user rhtyd commented on the issue:

https://github.com/apache/cloudstack/pull/1974
  
@dmabry yes you may close this, I'll cherry pick this on 4.9 from the 
merged commit on master.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1961: Fix for test_snapshots.py using nfs2 instead of nfs ...

[blueorangutan]

Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1961
  
@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been 
kicked to run smoke tests


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1961: Fix for test_snapshots.py using nfs2 instead of nfs ...

[rhtyd]

Github user rhtyd commented on the issue:

https://github.com/apache/cloudstack/pull/1961
  
@blueorangutan test


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: :[VOTE] Apache Cloudstack 4.10.0.0

[Rohit Yadav]

-1 (binding)


All, I've found an upgrade blocker. Pre 4.6 users are required to seed 4.6 
systemvmtemplate to proceed with the upgrade otherwise upgrade fails, and from 
4.9 upgrade to 4.10 does no check/enforcement that 4.10 based systemvmtemplate 
has been seeded/registered, nor the minimum required systemvmtemplate version 
is changed from 4.6.0 to 4.10.0.


After we have merged the strongswan/java8 PR, I had updated the upgrade docs on 
how to upgrade the systemvmtemplate here:

http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.10/upgrade/upgrade-4.9.html


Using the above, I've tried to fix these issues here, please review and merge 
for RC2:

https://github.com/apache/cloudstack/pull/1983


With above fix, the aim is that 
users only seed the 4.10 systemvmtemplate before upgrade and post-upgrade the 
upgrade paths fix the entries, global setting etc.


Regards.


From: Tutkowski, Mike 
Sent: 02 March 2017 22:39:08
To: dev@cloudstack.apache.org
Subject: Re: :[VOTE] Apache Cloudstack 4.10.0.0

I rolled back to my master branch at da66b06e7d562393da2e4b52206943f8bad49d10 
and it works.

It appears something that went into after that commit has broken this. It looks 
like this SHA is about two weeks old and that 43 commits have gone into master 
since it.

On 3/2/17, 7:06 AM, "Tutkowski, Mike"  wrote:

According to where the code fails, though, it appears to be a networking 
problem. If I set a breakpoint before the failure and change a variable to say 
that security groups are not being used, then the VM starts.

I think this is a recently introduced problem because I have another branch 
based off of a slightly older version of master and it works fine here.

> On Mar 2, 2017, at 6:51 AM, Pierre-Luc Dion  wrote:
>
> Hi Mike,
> Try vm with at least 512MB for memory.
>
>> On Mar 1, 2017 15:01, "Tutkowski, Mike"  
wrote:
>>
>> I see the following exception when trying to deploy a user VM in a Basic
>> Zone with two XenServer 6.5 hosts in one cluster. My system VMs have all
>> deployed properly. The user template gets downloaded fine. I can see the
>> user VM begin to start on a XenServer host, then it goes away. We then
>> automatically try on the other host. I can see the VM begin to start 
there
>> for a moment, then it goes away.
>>
>> I am just deploying the user VM’s template and root disk to NFS (same
>> place where the template and root disks of my system VMs are).
>>
>> I am using the built-in XenServer CentOS 5.6 (64 bit) template with 1
>> vCPU, 500 MHz, and 256 MB memory.
>>
>> WARN  [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-7:ctx-35aded78)
>> (logid:aab9c320) Expected 1 answers while executing VmDataCommand but
>> received 2
>> WARN  [c.c.v.VirtualMachinePowerStateSyncImpl] 
(DirectAgentCronJob-14:ctx-27fb1ac3)
>> (logid:2c342f23) VM state was updated but update time is null?! vm id: 6
>> INFO  [o.a.c.f.j.i.AsyncJobManagerImpl] 
(AsyncJobMgr-Heartbeat-1:ctx-2c7d2dce)
>> (logid:a56a9a8c) Begin cleanup expired async-jobs
>> INFO  [o.a.c.f.j.i.AsyncJobManagerImpl] 
(AsyncJobMgr-Heartbeat-1:ctx-2c7d2dce)
>> (logid:a56a9a8c) End cleanup expired async-jobs
>> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 removed accounts to cleanup
>> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 disabled accounts to cleanup
>> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 inactive domains to cleanup
>> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c)
>> (logid:541e9ba5) Found 0 disabled projects to cleanup
>> WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-16:ctx-7c901443)
>> (logid:aab9c320) callHostPlugin failed for cmd: default_network_rules 
with
>> args secIps: 0:, vmName: i-2-6-VM, vmID: 6, vmIP: 10.117.40.53, vmMAC:
>> 06:b2:f4:00:00:22,  due to There was a failure communicating with the
>> plugin.
>> WARN  [c.c.h.x.r.w.x.CitrixStartCommandWrapper]
>> (DirectAgent-16:ctx-7c901443) (logid:aab9c320) Catch Exception: class
>> com.cloud.utils.exception.CloudRuntimeException due to
>> com.cloud.utils.exception.CloudRuntimeException: callHostPlugin failed
>> for cmd: default_network_rules with args secIps: 0:, vmName: i-2-6-VM,
>> vmID: 6, vmIP: 10.117.40.53, vmMAC: 06:b2:f4:00:00:22,  due to There was 
a
>> failure communicating with the plugin.
>> com.cloud.utils.exception.CloudRuntimeException: callHostPlugin failed
>> for cmd: default_network_rules with args secIps: 0:, vmName: i-2-6-VM,
>> vmID: 6, vmIP: 10.117.40.53, vmMAC: 

[GitHub] cloudstack issue #1983: [4.10/blocker] Upgrade4920to41000: Migrate systemvmt...

[blueorangutan]

Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1983
  
@rhtyd a Jenkins job has been kicked to build packages. I'll keep you 
posted as I make progress.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1983: [4.10/blocker] Upgrade4920to41000: Migrate sy...

[rhtyd]

GitHub user rhtyd opened a pull request:

https://github.com/apache/cloudstack/pull/1983

[4.10/blocker] Upgrade4920to41000: Migrate systemvmtemplate for 4.10

- Adds systemvm migration for upgrading to 4.10
- Remove systemvm migration block for upgrading from 4.5 or below, this
  ensures that pre 4.6 users are not required to seed 4.6 systemvmtemplate
  to upgrade to 4.10, but can directly seed a 4.10 systemvmtemplate

Pinging for review - @karuturi @PaulAngus @abhinandanprateek @DaanHoogland 
@koushik-das @borisstoyanov @wido and others.

Note: for the 4.10 release systemvmtemplates are not available on 
http://cloudstack.apt-get.eu/systemvm yet so I've used URLs from 
packages.shapeblue.com.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/shapeblue/cloudstack 
4.10-systemvmtemplate-upgrade-fix

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1983.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1983


commit 0cec5f57067a43478db37074620727d105467d77
Author: Rohit Yadav 
Date:   2017-03-03T08:16:45Z

Upgrade4920to41000: Migrate systemvmtemplate for 4.10

- Adds systemvm migration for upgrading to 4.10
- Remove systemvm migration block for upgrading from 4.5 or below, this
  ensures that pre 4.6 users are not required to seed 4.6 systemvmtemplate
  to upgrade to 4.10, but can directly seed a 4.10 systemvmtemplate

Signed-off-by: Rohit Yadav 




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1983: [4.10/blocker] Upgrade4920to41000: Migrate systemvmt...

[rhtyd]

Github user rhtyd commented on the issue:

https://github.com/apache/cloudstack/pull/1983
  
@blueorangutan package


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1894: CLOUDSTACK-9700 Allow user to Register/Copy t...

[priyankparihar]

Github user priyankparihar commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1894#discussion_r104106536
  
--- Diff: 
api/src/org/apache/cloudstack/api/command/admin/template/RegisterTemplateCmdByAdmin.java
 ---
@@ -40,10 +40,23 @@
 @Override
 public void execute() throws ResourceAllocationException{
 try {
+if ((zoneId != null) && (zoneIds != null && 
!zoneIds.isEmpty()))
--- End diff --

Unnecessary brackets should be removed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1961: Fix for test_snapshots.py using nfs2 instead of nfs ...

[blueorangutan]

Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1961
  
Packaging result: ✔centos6 ✔centos7 ✔debian. JID-559


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1894: CLOUDSTACK-9700 Allow user to Register/Copy t...

[priyankparihar]

Github user priyankparihar commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1894#discussion_r104105564
  
--- Diff: 
api/src/org/apache/cloudstack/api/command/admin/template/CopyTemplateCmdByAdmin.java
 ---
@@ -40,11 +40,20 @@
 @Override
 public void execute() throws ResourceAllocationException{
 try {
+if (destZoneId == null && (destZoneIds == null || 
destZoneIds.size() == 0))
+throw new ServerApiException(ApiErrorCode.PARAM_ERROR,
+"Either destzoneid or destzoneids parameters have 
to be specified.");
+
--- End diff --

Error statement and conditional statement logic are not conveying same 
message. Some correction is required.  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1813: CLOUDSTACK-9604: Root disk resize support for VMware...

[blueorangutan]

Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1813
  
Packaging result: ✔centos6 ✔centos7 ✔debian. JID-558


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1894: CLOUDSTACK-9700 Allow user to Register/Copy t...

[priyankparihar]

Github user priyankparihar commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1894#discussion_r104105003
  
--- Diff: api/src/org/apache/cloudstack/api/ResponseGenerator.java ---
@@ -307,7 +307,11 @@
 
 TemplateResponse createTemplateUpdateResponse(ResponseView view, 
VirtualMachineTemplate result);
 
-List createTemplateResponses(ResponseView view, 
VirtualMachineTemplate result, Long zoneId, boolean readyOnly);
+List createTemplateResponses(ResponseView view, 
VirtualMachineTemplate result,
+   Long zoneId, boolean 
readyOnly);
--- End diff --

what is difference between line 310 and 311 ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---