[jira] [Closed] (CLOUDSTACK-10312) keealived.log will end up filling ramdisk partition on RVR, As timestamp comparison is wrong

[jay (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

jay closed CLOUDSTACK-10312.

Resolution: Fixed

duplicate hence closing the issue

> keealived.log will end up filling ramdisk partition on RVR, As timestamp 
> comparison is wrong 
> -
>
> Key: CLOUDSTACK-10312
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10312
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Virtual Router
>Affects Versions: 4.6.0, 4.11.0.0
>Reporter: jay
>Priority: Major
>
> coding error in how the time stamp is compared in check_heartbeat.sh file .
> thistime=$(cat $ROUTER_BIN_PATH/keepalived.ts)
>  lasttime=$(cat $ROUTER_BIN_PATH/keepalived.ts2)
>  diff=$(($lasttime - $thistime))
> s=0
>  if  [ $diff -ge 10 ]
> lasttime will never be greater then current time and IF condition will always 
> fail
> related commit ID :
> https://github.com/apache/cloudstack/pull/587/files#diff-002a85328505a
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (CLOUDSTACK-10237) L2 networks should be allowed to be shared and used in projects

[Nicolas Vazquez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicolas Vazquez resolved CLOUDSTACK-10237.
--
   Resolution: Fixed
Fix Version/s: 4.11.1.0

> L2 networks should be allowed to be shared and used in projects
> ---
>
> Key: CLOUDSTACK-10237
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10237
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.11.0.0
> Environment: Centos7, KVM, advanced zones with vlan based isolation
>Reporter: Jean-Francois Nadeau
>Assignee: Nicolas Vazquez
>Priority: Major
> Fix For: 4.11.1.0
>
>
> Hi all,
>  
> I'm testing 4.11-rc1 and the new L2 network type feature as shown at 
> [http://www.shapeblue.com/layer-2-networks-in-cloudstack/]
>  
> I want to use this as a replacement to a shared network offering with no DHCP 
> which works to support an external DHCP server but still required to fill 
> some CIDR information.
>  
> I thought the intent was that L2 network type was to replace the previous 
> approach when required to integrate an existing network/DHCP.
>  
> If I attempt to provision a VM in a project as the root admin using the L2 
> network I get denied... apparently because they are not shared and I can't 
> make them public.
>  
> ('HTTP 531 response from CloudStack', , \{u'errorcode': 531, 
> u'uuidList': [], u'cserrorcode': 4365, u'errortext': u'Unable to use network 
> with id= 7712102b-bbdf-4c54-bdbf-9fddfa16de46, permission denied'}) 
>  
> Provisioning in the root project works just fine  but really I want to 
> use L2 networks in user projects even if only the admin can do so.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (CLOUDSTACK-10231) Asserted fixes for Direct Download on KVM

[Nicolas Vazquez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicolas Vazquez resolved CLOUDSTACK-10231.
--
Resolution: Fixed

> Asserted fixes for Direct Download on KVM
> -
>
> Key: CLOUDSTACK-10231
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10231
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: KVM
>Affects Versions: 4.11.0.0
>Reporter: Nicolas Vazquez
>Assignee: Nicolas Vazquez
>Priority: Major
>  Labels: direct-download,, kvm
> Fix For: 4.11.1.0
>
>
> Several fixes addressed:
>  * Dettach ISO fails when trying to detach a direct download ISO
>  * Fix for metalink support on SSVM agents (this closes CLOUDSTACK-10238)
>  * Reinstall VM from bypassed registered template (this closes 
> CLOUDSTACK-10250)
>  * Fix upload certificate error message even though operation was successful
>  * Fix metalink download, checksum retry logic and metalink SSVM downloader



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (CLOUDSTACK-10146) Bypass Secondary Storage for KVM templates

[Nicolas Vazquez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicolas Vazquez resolved CLOUDSTACK-10146.
--
   Resolution: Fixed
Fix Version/s: 4.11.0.0

> Bypass Secondary Storage for KVM templates
> --
>
> Key: CLOUDSTACK-10146
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10146
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.11.0.0
>Reporter: Nicolas Vazquez
>Assignee: Nicolas Vazquez
>Priority: Major
> Fix For: 4.11.0.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (CLOUDSTACK-10231) Asserted fixes for Direct Download on KVM

[Nicolas Vazquez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicolas Vazquez updated CLOUDSTACK-10231:
-
Status: Reviewable  (was: In Progress)

> Asserted fixes for Direct Download on KVM
> -
>
> Key: CLOUDSTACK-10231
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10231
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: KVM
>Affects Versions: 4.11.0.0
>Reporter: Nicolas Vazquez
>Assignee: Nicolas Vazquez
>Priority: Major
>  Labels: direct-download,, kvm
> Fix For: 4.11.1.0
>
>
> Several fixes addressed:
>  * Dettach ISO fails when trying to detach a direct download ISO
>  * Fix for metalink support on SSVM agents (this closes CLOUDSTACK-10238)
>  * Reinstall VM from bypassed registered template (this closes 
> CLOUDSTACK-10250)
>  * Fix upload certificate error message even though operation was successful
>  * Fix metalink download, checksum retry logic and metalink SSVM downloader



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (CLOUDSTACK-10109) Enable dedication of public IPs to SSVM and CPVM

[Nicolas Vazquez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicolas Vazquez resolved CLOUDSTACK-10109.
--
   Resolution: Fixed
Fix Version/s: 4.11.0.0

> Enable dedication of public IPs to SSVM and CPVM
> 
>
> Key: CLOUDSTACK-10109
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10109
> Project: CloudStack
>  Issue Type: New Feature
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Reporter: Nicolas Vazquez
>Assignee: Nicolas Vazquez
>Priority: Major
> Fix For: 4.11.0.0
>
> Attachments: public01.png, public02.png, public03.png
>
>
> It is required to dedicate a public IP range for SSVM and CPVM in order to 
> apply firewall rules to control inbound access.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (CLOUDSTACK-10126) Separate Subnet for CPVM and SSVM

[Nicolas Vazquez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicolas Vazquez resolved CLOUDSTACK-10126.
--
   Resolution: Fixed
Fix Version/s: 4.11.0.0

> Separate Subnet for CPVM and SSVM
> -
>
> Key: CLOUDSTACK-10126
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10126
> Project: CloudStack
>  Issue Type: New Feature
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.11.0.0
>Reporter: Nicolas Vazquez
>Assignee: Nicolas Vazquez
>Priority: Major
> Fix For: 4.11.0.0
>
>
> Separate Management Subnet for CPVM and SSVM



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (CLOUDSTACK-10251) HTTPS downloader for Direct Download templates failure

[Nicolas Vazquez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicolas Vazquez resolved CLOUDSTACK-10251.
--
   Resolution: Fixed
Fix Version/s: 4.11.1.0

> HTTPS downloader for Direct Download templates failure 
> ---
>
> Key: CLOUDSTACK-10251
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10251
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.11.0.0
>Reporter: Nicolas Vazquez
>Assignee: Nicolas Vazquez
>Priority: Major
> Fix For: 4.11.1.0
>
>
> Failure on HTTPS downloader for Direct Download templates on KVM.
> Reason: Incorrect request caused NullPointerException getting the response 
> InputStream



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (CLOUDSTACK-10247) L2 network not shared on projects

[Nicolas Vazquez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicolas Vazquez resolved CLOUDSTACK-10247.
--
   Resolution: Fixed
Fix Version/s: 4.11.1.0

> L2 network not shared on projects
> -
>
> Key: CLOUDSTACK-10247
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10247
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Reporter: Nicolas Vazquez
>Assignee: Nicolas Vazquez
>Priority: Major
> Fix For: 4.11.1.0
>
>
> L2 networks are not shared between projects. When trying to deploy a vm 
> assigning a project id, error is logged:
> 2018-01-18 09:04:31,749 INFO  [c.c.a.ApiServer] 
> (qtp1310540333-17:ctx-7ff91c56 ctx-8c37c46b ctx-d58064ad) (logid:74e86028) 
> PermissionDenied: Unable to use network with id= 
> 5bee486a-ff20-4db2-b62e-4b4f3485cfff, permission denied on objs: []



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (CLOUDSTACK-10321) CPU Cap for KVM

[Nicolas Vazquez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicolas Vazquez resolved CLOUDSTACK-10321.
--
   Resolution: Fixed
Fix Version/s: 4.11.1.0

Merged into 4.11.1.0+ milestone

> CPU Cap for KVM
> ---
>
> Key: CLOUDSTACK-10321
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10321
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Reporter: Nicolas Vazquez
>Assignee: Nicolas Vazquez
>Priority: Major
> Fix For: 4.11.1.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10323) Change disk offering when volume is migrated to different type of storage pool.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405620#comment-16405620
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10323:
-

blueorangutan commented on issue #2486: [CLOUDSTACK-10323] Allow changing disk 
offering during volume migration 
URL: https://github.com/apache/cloudstack/pull/2486#issuecomment-374426739
 
 
   Trillian test result (tid-2393)
   Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 32550 seconds
   Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2486-t2393-kvm-centos7.zip
   Intermitten failure detected: /marvin/tests/smoke/test_certauthority_root.py
   Intermitten failure detected: 
/marvin/tests/smoke/test_deploy_virtio_scsi_vm.py
   Intermitten failure detected: /marvin/tests/smoke/test_primary_storage.py
   Intermitten failure detected: /marvin/tests/smoke/test_public_ip_range.py
   Intermitten failure detected: /marvin/tests/smoke/test_snapshots.py
   Intermitten failure detected: /marvin/tests/smoke/test_templates.py
   Intermitten failure detected: /marvin/tests/smoke/test_usage.py
   Intermitten failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
   Intermitten failure detected: /marvin/tests/smoke/test_volumes.py
   Intermitten failure detected: /marvin/tests/smoke/test_vpc_redundant.py
   Intermitten failure detected: /marvin/tests/smoke/test_host_maintenance.py
   Intermitten failure detected: /marvin/tests/smoke/test_hostha_kvm.py
   Smoke tests completed. 58 look OK, 9 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   test_01_add_primary_storage_disabled_host | `Error` | 0.71 | 
test_primary_storage.py
   test_01_primary_storage_nfs | `Error` | 0.11 | test_primary_storage.py
   ContextSuite context=TestStorageTags>:setup | `Error` | 0.19 | 
test_primary_storage.py
   test_02_list_snapshots_with_removed_data_store | `Error` | 1.15 | 
test_snapshots.py
   test_04_extract_template | `Failure` | 128.28 | test_templates.py
   ContextSuite context=TestISOUsage>:setup | `Error` | 0.00 | test_usage.py
   test_08_migrate_vm | `Error` | 15.71 | test_vm_life_cycle.py
   test_06_download_detached_volume | `Failure` | 137.56 | test_volumes.py
   test_04_rvpc_network_garbage_collector_nics | `Failure` | 448.41 | 
test_vpc_redundant.py
   test_01_cancel_host_maintenace_with_no_migration_jobs | `Failure` | 0.09 | 
test_host_maintenance.py
   test_02_cancel_host_maintenace_with_migration_jobs | `Error` | 2.23 | 
test_host_maintenance.py
   test_hostha_enable_ha_when_host_in_maintenance | `Error` | 2.42 | 
test_hostha_kvm.py
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Change disk offering when volume is migrated to different type of storage 
> pool.
> ---
>
> Key: CLOUDSTACK-10323
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10323
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.12
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> This is a continuation of work developed on PR #2425 (CLOUDSTACK-10240), 
> which provided root admins an override mechanism to move volumes between 
> storage systems types (local/shared) even when the disk offering would not 
> allow such operation. To complete the work, we will now provide a way for 
> administrators to enter a new disk offering that can reflect the new 
> placement of the volume. We will add an extra parameter to allow the root 
> admin inform a new disk offering for the volume. Therefore, when the volume 
> is being migrated, it will be possible to replace the disk offering to 
> reflect the new placement of the volume.
> The API method will have the following parameters: 
> * storageid (required)
> * volumeid (required)
> * livemigrate(optional)
> * newdiskofferingid (optional) – this is the new parameter
> The expected behavior is the following: 
> * If “newdiskofferingid” is not provided the current behavior is maintained. 
> Override mechanism will also keep working as we have seen so far. 
> * If the “newdiskofferingid” is provided by the admin, we will execute the 
> following checks
> ** new disk offering mode (local/shared) must match the target storage mode. 
> If it does not match, an exception will be thrown and the operator will 
> receive a message 

[jira] [Commented] (CLOUDSTACK-10226) CloudStack is not importing Local storage properly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405502#comment-16405502
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10226:
-

blueorangutan commented on issue #2401: [CLOUDSTACK-10226] CloudStack is not 
importing Local storage properly
URL: https://github.com/apache/cloudstack/pull/2401#issuecomment-374391259
 
 
   Trillian test result (tid-2395)
   Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 21881 seconds
   Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2401-t2395-xenserver-71.zip
   Intermitten failure detected: /marvin/tests/smoke/test_certauthority_root.py
   Intermitten failure detected: /marvin/tests/smoke/test_scale_vm.py
   Intermitten failure detected: /marvin/tests/smoke/test_service_offerings.py
   Intermitten failure detected: /marvin/tests/smoke/test_vpc_redundant.py
   Smoke tests completed. 64 look OK, 3 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   test_01_scale_vm | `Error` | 10.33 | test_scale_vm.py
   ContextSuite context=TestCpuCapServiceOfferings>:teardown | `Error` | 0.00 | 
test_service_offerings.py
   test_04_rvpc_network_garbage_collector_nics | `Failure` | 507.42 | 
test_vpc_redundant.py
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> CloudStack is not importing Local storage properly 
> ---
>
> Key: CLOUDSTACK-10226
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10226
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: XenServer
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
> Fix For: 4.12
>
>
> CloudStack is importing as Local storage any XenServer SR that is of type LVM 
> or EXT. This causes a problem when one wants to use both Direct attach 
> storage and local storage. Moreover, CloudStack was not importing all of the 
> local storage that a host has available when local storage is enabled. It was 
> only importing the First SR it sees.
> To fix the first problem we started ignoring SRs that have the flag 
> shared=true when discovering local storages. SRs configured to be shared are 
> used as direct attached storage, and therefore should not be imported again 
> as local ones.
> To fix the second problem, we started loading all Local storage and importing 
> them accordingly to ACS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10326) Prevent hosts fall into Maintenance when there are running VMs on it

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405228#comment-16405228
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10326:
-

blueorangutan commented on issue #2493: CLOUDSTACK-10326: Prevent hosts fall 
into Maintenance when there are running VMs on it
URL: https://github.com/apache/cloudstack/pull/2493#issuecomment-374316850
 
 
   Trillian test result (tid-2391)
   Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
   Total time taken: 24551 seconds
   Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2493-t2391-kvm-centos7.zip
   Intermitten failure detected: 
/marvin/tests/smoke/test_deploy_virtio_scsi_vm.py
   Intermitten failure detected: /marvin/tests/smoke/test_ssvm.py
   Intermitten failure detected: /marvin/tests/smoke/test_vpc_redundant.py
   Intermitten failure detected: /marvin/tests/smoke/test_host_maintenance.py
   Intermitten failure detected: /marvin/tests/smoke/test_hostha_kvm.py
   Smoke tests completed. 64 look OK, 3 have error(s)
   Only failed tests results shown below:
   
   
   Test | Result | Time (s) | Test File
   --- | --- | --- | ---
   test_04_rvpc_network_garbage_collector_nics | `Failure` | 498.07 | 
test_vpc_redundant.py
   test_02_cancel_host_maintenace_with_migration_jobs | `Error` | 78.50 | 
test_host_maintenance.py
   test_hostha_enable_ha_when_host_in_maintenance | `Error` | 3.81 | 
test_hostha_kvm.py
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Prevent hosts fall into Maintenance when there are running VMs on it
> 
>
> Key: CLOUDSTACK-10326
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10326
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.11.0.0
>Reporter: Nicolas Vazquez
>Assignee: Nicolas Vazquez
>Priority: Major
> Fix For: 4.11.1.0
>
> Attachments: CLOUDSTACK-10326-Debug.png, 
> CLOUDSTACK-10326-InitialState.png, CLOUDSTACK-10326-Migrating.png, 
> CLOUDSTACK-10326-MigrationFailed.png
>
>
> This issue was discovered, fixed and tested on KVM, but applies for every 
> hypervisor.
> h2. Background
> When enabling maintenance mode in a host, host state is put into 
> 'PrepareForMaintenance' and running VMs are migrated into another host. After 
> every VM is migrated, host goes to 'Maintenance' state.
> Checks are performed on ResourceManagerImpl.checkAndMaintan() method:
>  * List VMs with host_id = HOST_ID
>  * List VMs with last_host_id = HOST_ID and state=Migrating
> When both queries are empty, then the host can be put into Maintenance.
> When a VM is being migrated to DEST_HOST, its host_id column is set to 
> DEST_HOST, last_host_id = ORIGIN_HOST and state = Migrating. If then 
> migration fails, host_id = last_host_id = ORIGIN_HOST 
> h2. Issue
> This sequence:
>  * Enable maintenance mode on ORIGIN_HOST
>  * VMs start being migrated to a host, say DEST_HOST
>  * checkAndMaintain() starts:
>  ** First check passes (no VM with host_id = ORIGIN_HOST_ID as those are 
> being migrated)
>  ** Before the second check, one or more migrations fail
>  ** Second check passes, however there are VMs running on the host as 
> migrations have failed.
>  * Host goes into Maintenance state.
> Screenshots attached, query executed on each case:
> select id, name, instance_name, state, host_id, last_host_id from vm_instance;



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10320) Invalid pair for response object breaking response parsing

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405057#comment-16405057
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10320:
-

marcaurele commented on issue #2481: CLOUDSTACK-10320 - Invalid pair for 
response object breaking response parsing
URL: https://github.com/apache/cloudstack/pull/2481#issuecomment-374269828
 
 
   The default isolation level in CS is `read-committed` 
(https://github.com/apache/cloudstack/blob/master/framework/db/src/main/java/com/cloud/utils/db/TransactionLegacy.java#L1044)
 but not `repeatable-read`, except if you explicitly set it in the 
`db.properties` file. If the default value is change, and I modify the PR to 
use a transaction around those 2 SQL read operations, the issue is fixed, but...
   
   Changing the default isolation level will certainly have an impact on 
performance on the database 
(https://www.percona.com/blog/2015/01/14/mysql-performance-implications-of-innodb-isolation-modes/).
 So, I'll change the PR to only have a repeatable-read isolation for those set 
of methods, and will try to measure how it performs.
   
   If people are willing to test in their environment (quite loaded) the global 
isolation level change, it would be interesting to hear the performance 
outcomes.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Invalid pair for response object breaking response parsing
> --
>
> Key: CLOUDSTACK-10320
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10320
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: API
>Reporter: Marc-Aurèle Brothier
>Assignee: Marc-Aurèle Brothier
>Priority: Major
>
> Under some circumstances, the API is returning an invalid response, for 
> simplicity I will expose the JSON case. The API response on a 
> listVirtualMachines can be this string:
> {code:java}
> { "listvirtualmachinesresponse" :  ] } }{code}
> To understand how this is possible, assume you have more than one management 
> server and one is processing the destroy of a virtual machine in the account 
> X which is the only one it has. Another process is returning the result of 
> listVirtualMachines for that same account X. During the listVM command, the 
> result set is fetch with a searchAndDistinctCount due to the view 
> ([https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/query/QueryManagerImpl.java#L1024).]
>  This is done through 2 queries in the GenericDao 
> [https://github.com/apache/cloudstack/blob/master/framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java#L1323]
>  and if you encounter the _right_ conditions, the VM will be marked as 
> removed in between those 2 queries. This results in having a Pair result with 
> at least one object but a count of 0. Then following how is done the 
> serialization of the response at 
> [https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/response/ApiResponseSerializer.java#L86]
>  you will reach the case where your output is the one previously mentioned.
> To overcome this issue, there isn't a true fix but only a better pair 
> response to ensure a correct response formatting. If the result set contains 
> at least something, the count cannot be 0 but we cannot guess the correct 
> answer, but only state it has at least one element.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10226) CloudStack is not importing Local storage properly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404956#comment-16404956
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10226:
-

blueorangutan commented on issue #2401: [CLOUDSTACK-10226] CloudStack is not 
importing Local storage properly
URL: https://github.com/apache/cloudstack/pull/2401#issuecomment-374248526
 
 
   @DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + xenserver-71) has 
been kicked to run smoke tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> CloudStack is not importing Local storage properly 
> ---
>
> Key: CLOUDSTACK-10226
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10226
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: XenServer
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
> Fix For: 4.12
>
>
> CloudStack is importing as Local storage any XenServer SR that is of type LVM 
> or EXT. This causes a problem when one wants to use both Direct attach 
> storage and local storage. Moreover, CloudStack was not importing all of the 
> local storage that a host has available when local storage is enabled. It was 
> only importing the First SR it sees.
> To fix the first problem we started ignoring SRs that have the flag 
> shared=true when discovering local storages. SRs configured to be shared are 
> used as direct attached storage, and therefore should not be imported again 
> as local ones.
> To fix the second problem, we started loading all Local storage and importing 
> them accordingly to ACS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10226) CloudStack is not importing Local storage properly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404955#comment-16404955
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10226:
-

DaanHoogland commented on issue #2401: [CLOUDSTACK-10226] CloudStack is not 
importing Local storage properly
URL: https://github.com/apache/cloudstack/pull/2401#issuecomment-374248408
 
 
   @blueorangutan test centos7 xenserver-71


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> CloudStack is not importing Local storage properly 
> ---
>
> Key: CLOUDSTACK-10226
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10226
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: XenServer
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
> Fix For: 4.12
>
>
> CloudStack is importing as Local storage any XenServer SR that is of type LVM 
> or EXT. This causes a problem when one wants to use both Direct attach 
> storage and local storage. Moreover, CloudStack was not importing all of the 
> local storage that a host has available when local storage is enabled. It was 
> only importing the First SR it sees.
> To fix the first problem we started ignoring SRs that have the flag 
> shared=true when discovering local storages. SRs configured to be shared are 
> used as direct attached storage, and therefore should not be imported again 
> as local ones.
> To fix the second problem, we started loading all Local storage and importing 
> them accordingly to ACS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10226) CloudStack is not importing Local storage properly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404951#comment-16404951
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10226:
-

DaanHoogland commented on issue #2401: [CLOUDSTACK-10226] CloudStack is not 
importing Local storage properly
URL: https://github.com/apache/cloudstack/pull/2401#issuecomment-374247491
 
 
   @blueorangutan help


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> CloudStack is not importing Local storage properly 
> ---
>
> Key: CLOUDSTACK-10226
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10226
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: XenServer
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
> Fix For: 4.12
>
>
> CloudStack is importing as Local storage any XenServer SR that is of type LVM 
> or EXT. This causes a problem when one wants to use both Direct attach 
> storage and local storage. Moreover, CloudStack was not importing all of the 
> local storage that a host has available when local storage is enabled. It was 
> only importing the First SR it sees.
> To fix the first problem we started ignoring SRs that have the flag 
> shared=true when discovering local storages. SRs configured to be shared are 
> used as direct attached storage, and therefore should not be imported again 
> as local ones.
> To fix the second problem, we started loading all Local storage and importing 
> them accordingly to ACS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10226) CloudStack is not importing Local storage properly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404953#comment-16404953
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10226:
-

blueorangutan commented on issue #2401: [CLOUDSTACK-10226] CloudStack is not 
importing Local storage properly
URL: https://github.com/apache/cloudstack/pull/2401#issuecomment-374247738
 
 
   @DaanHoogland I understand these words: "help", "hello", "thanks", 
"package", "test"
   Test command usage: test [mgmt os] [hypervisor] [additional tests]
   Mgmt OS options: ['centos6', 'centos7', 'ubuntu']
   Hypervisor options: ['kvm-centos6', 'kvm-centos7', 'kvm-ubuntu', 
'xenserver-71', 'xenserver-65sp1', 'xenserver-62sp1', 'vmware-65', 
'vmware-60u2', 'vmware-55u3', 'vmware-51u1', 'vmware-50u1']
   Additional tests: list of space separated tests with paths relative to the 
`test/integration` directory, for example: component/test_acl_listvm.py 
component/test_volumes.py
   Note: when additional tests are passed, you need to specify mgmt server os 
and hypervisor or use the `matrix` command.
   
   Blessed contributors for kicking Trillian test jobs: ['rhtyd', 'nvazquez', 
'PaulAngus', 'borisstoyanov', 'DaanHoogland']


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> CloudStack is not importing Local storage properly 
> ---
>
> Key: CLOUDSTACK-10226
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10226
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: XenServer
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
> Fix For: 4.12
>
>
> CloudStack is importing as Local storage any XenServer SR that is of type LVM 
> or EXT. This causes a problem when one wants to use both Direct attach 
> storage and local storage. Moreover, CloudStack was not importing all of the 
> local storage that a host has available when local storage is enabled. It was 
> only importing the First SR it sees.
> To fix the first problem we started ignoring SRs that have the flag 
> shared=true when discovering local storages. SRs configured to be shared are 
> used as direct attached storage, and therefore should not be imported again 
> as local ones.
> To fix the second problem, we started loading all Local storage and importing 
> them accordingly to ACS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10323) Change disk offering when volume is migrated to different type of storage pool.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404915#comment-16404915
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10323:
-

blueorangutan commented on issue #2486: [CLOUDSTACK-10323] Allow changing disk 
offering during volume migration 
URL: https://github.com/apache/cloudstack/pull/2486#issuecomment-374233841
 
 
   @borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has 
been kicked to run smoke tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Change disk offering when volume is migrated to different type of storage 
> pool.
> ---
>
> Key: CLOUDSTACK-10323
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10323
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.12
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> This is a continuation of work developed on PR #2425 (CLOUDSTACK-10240), 
> which provided root admins an override mechanism to move volumes between 
> storage systems types (local/shared) even when the disk offering would not 
> allow such operation. To complete the work, we will now provide a way for 
> administrators to enter a new disk offering that can reflect the new 
> placement of the volume. We will add an extra parameter to allow the root 
> admin inform a new disk offering for the volume. Therefore, when the volume 
> is being migrated, it will be possible to replace the disk offering to 
> reflect the new placement of the volume.
> The API method will have the following parameters: 
> * storageid (required)
> * volumeid (required)
> * livemigrate(optional)
> * newdiskofferingid (optional) – this is the new parameter
> The expected behavior is the following: 
> * If “newdiskofferingid” is not provided the current behavior is maintained. 
> Override mechanism will also keep working as we have seen so far. 
> * If the “newdiskofferingid” is provided by the admin, we will execute the 
> following checks
> ** new disk offering mode (local/shared) must match the target storage mode. 
> If it does not match, an exception will be thrown and the operator will 
> receive a message indicating the problem.
> ** we will check if the new disk offering tags match the target storage tags. 
> If it does not match, an exception will be thrown and the operator will 
> receive a message indicating the problem.
> ** check if the target storage has the capacity for the new volume. If it 
> does not have enough space, then an exception is thrown and the operator will 
> receive a message indicating the problem.
> ** check if the size of the volume is the same as the size of the new disk 
> offering. If it is not the same, we will ALLOW the change of the service 
> offering, and a warning message will be logged.
> We execute the change of the Disk offering as soon as the migration of the 
> volume finishes. Therefore, if an error happens during the migration and the 
> volume remains in the original storage system, the disk offering will keep 
> reflecting this situation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10214) Unable to remove local primary storage

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404912#comment-16404912
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10214:
-

borisstoyanov commented on issue #2390: [CLOUDSTACK-10214] Unable to remove 
local primary storage
URL: https://github.com/apache/cloudstack/pull/2390#issuecomment-374233545
 
 
   I've just re-run it and it succeed. @rafaelweingartner 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Unable to remove local primary storage 
> ---
>
> Key: CLOUDSTACK-10214
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10214
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.10.0.0, 4.9.3.0
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> When enabling the use of local storage ACS will automatically load all local 
> storage configured in the Host and start using them as primary storage to 
> deploy user VMs (if the service offering allows to do so). However, if the 
> operator wants to remove the local storage ACS will throw an exception saying 
> that the removal of local storage is not allowed.Therefore, if one wants to 
> remove a local storage, he/she needs to do a manual intervention in the 
> database and hosts.
> This limitation was removed, as it was only a logical restriction.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10323) Change disk offering when volume is migrated to different type of storage pool.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404913#comment-16404913
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10323:
-

borisstoyanov commented on issue #2486: [CLOUDSTACK-10323] Allow changing disk 
offering during volume migration 
URL: https://github.com/apache/cloudstack/pull/2486#issuecomment-374233705
 
 
   @blueorangutan test


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Change disk offering when volume is migrated to different type of storage 
> pool.
> ---
>
> Key: CLOUDSTACK-10323
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10323
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.12
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> This is a continuation of work developed on PR #2425 (CLOUDSTACK-10240), 
> which provided root admins an override mechanism to move volumes between 
> storage systems types (local/shared) even when the disk offering would not 
> allow such operation. To complete the work, we will now provide a way for 
> administrators to enter a new disk offering that can reflect the new 
> placement of the volume. We will add an extra parameter to allow the root 
> admin inform a new disk offering for the volume. Therefore, when the volume 
> is being migrated, it will be possible to replace the disk offering to 
> reflect the new placement of the volume.
> The API method will have the following parameters: 
> * storageid (required)
> * volumeid (required)
> * livemigrate(optional)
> * newdiskofferingid (optional) – this is the new parameter
> The expected behavior is the following: 
> * If “newdiskofferingid” is not provided the current behavior is maintained. 
> Override mechanism will also keep working as we have seen so far. 
> * If the “newdiskofferingid” is provided by the admin, we will execute the 
> following checks
> ** new disk offering mode (local/shared) must match the target storage mode. 
> If it does not match, an exception will be thrown and the operator will 
> receive a message indicating the problem.
> ** we will check if the new disk offering tags match the target storage tags. 
> If it does not match, an exception will be thrown and the operator will 
> receive a message indicating the problem.
> ** check if the target storage has the capacity for the new volume. If it 
> does not have enough space, then an exception is thrown and the operator will 
> receive a message indicating the problem.
> ** check if the size of the volume is the same as the size of the new disk 
> offering. If it is not the same, we will ALLOW the change of the service 
> offering, and a warning message will be logged.
> We execute the change of the Disk offering as soon as the migration of the 
> volume finishes. Therefore, if an error happens during the migration and the 
> volume remains in the original storage system, the disk offering will keep 
> reflecting this situation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10214) Unable to remove local primary storage

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404852#comment-16404852
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10214:
-

rafaelweingartner commented on issue #2390: [CLOUDSTACK-10214] Unable to remove 
local primary storage
URL: https://github.com/apache/cloudstack/pull/2390#issuecomment-374220546
 
 
   @borisstoyanov is it possible to get some details on why the Debian package 
is failing here?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Unable to remove local primary storage 
> ---
>
> Key: CLOUDSTACK-10214
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10214
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.10.0.0, 4.9.3.0
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> When enabling the use of local storage ACS will automatically load all local 
> storage configured in the Host and start using them as primary storage to 
> deploy user VMs (if the service offering allows to do so). However, if the 
> operator wants to remove the local storage ACS will throw an exception saying 
> that the removal of local storage is not allowed.Therefore, if one wants to 
> remove a local storage, he/she needs to do a manual intervention in the 
> database and hosts.
> This limitation was removed, as it was only a logical restriction.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10214) Unable to remove local primary storage

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404849#comment-16404849
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10214:
-

rafaelweingartner commented on issue #2390: [CLOUDSTACK-10214] Unable to remove 
local primary storage
URL: https://github.com/apache/cloudstack/pull/2390#issuecomment-374220546
 
 
   @borisstoyanov is it possible to understand why the Debian package is 
failing here?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Unable to remove local primary storage 
> ---
>
> Key: CLOUDSTACK-10214
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10214
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.10.0.0, 4.9.3.0
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> When enabling the use of local storage ACS will automatically load all local 
> storage configured in the Host and start using them as primary storage to 
> deploy user VMs (if the service offering allows to do so). However, if the 
> operator wants to remove the local storage ACS will throw an exception saying 
> that the removal of local storage is not allowed.Therefore, if one wants to 
> remove a local storage, he/she needs to do a manual intervention in the 
> database and hosts.
> This limitation was removed, as it was only a logical restriction.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10214) Unable to remove local primary storage

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404824#comment-16404824
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10214:
-

blueorangutan commented on issue #2390: [CLOUDSTACK-10214] Unable to remove 
local primary storage
URL: https://github.com/apache/cloudstack/pull/2390#issuecomment-374217007
 
 
   Packaging result: ✔centos6 ✔centos7 ✖debian. JID-1793


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Unable to remove local primary storage 
> ---
>
> Key: CLOUDSTACK-10214
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10214
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.10.0.0, 4.9.3.0
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> When enabling the use of local storage ACS will automatically load all local 
> storage configured in the Host and start using them as primary storage to 
> deploy user VMs (if the service offering allows to do so). However, if the 
> operator wants to remove the local storage ACS will throw an exception saying 
> that the removal of local storage is not allowed.Therefore, if one wants to 
> remove a local storage, he/she needs to do a manual intervention in the 
> database and hosts.
> This limitation was removed, as it was only a logical restriction.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10226) CloudStack is not importing Local storage properly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404823#comment-16404823
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10226:
-

blueorangutan commented on issue #2401: [CLOUDSTACK-10226] CloudStack is not 
importing Local storage properly
URL: https://github.com/apache/cloudstack/pull/2401#issuecomment-374216584
 
 
   Packaging result: ✔centos6 ✔centos7 ✔debian. JID-1792


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> CloudStack is not importing Local storage properly 
> ---
>
> Key: CLOUDSTACK-10226
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10226
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: XenServer
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
> Fix For: 4.12
>
>
> CloudStack is importing as Local storage any XenServer SR that is of type LVM 
> or EXT. This causes a problem when one wants to use both Direct attach 
> storage and local storage. Moreover, CloudStack was not importing all of the 
> local storage that a host has available when local storage is enabled. It was 
> only importing the First SR it sees.
> To fix the first problem we started ignoring SRs that have the flag 
> shared=true when discovering local storages. SRs configured to be shared are 
> used as direct attached storage, and therefore should not be imported again 
> as local ones.
> To fix the second problem, we started loading all Local storage and importing 
> them accordingly to ACS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10230) User is able to change to “Guest OS type” that has been removed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404821#comment-16404821
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10230:
-

blueorangutan commented on issue #2404: [CLOUDSTACK-10230] User should not be 
able to use removed “Guest OS type”
URL: https://github.com/apache/cloudstack/pull/2404#issuecomment-374216198
 
 
   Packaging result: ✔centos6 ✔centos7 ✖debian. JID-1791


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> User is able to change to “Guest OS type” that has been removed 
> 
>
> Key: CLOUDSTACK-10230
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10230
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Critical
>
> Users are able to change the OS type of VMs to “Guest OS type” that has been 
> removed. This becomes a security issue when we try to force users to use HVM 
> VMs (Meltdown/Spectre thing). A removed “guest os type” should not be usable 
> by any users in the cloud.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10323) Change disk offering when volume is migrated to different type of storage pool.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404822#comment-16404822
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10323:
-

blueorangutan commented on issue #2486: [CLOUDSTACK-10323] Allow changing disk 
offering during volume migration 
URL: https://github.com/apache/cloudstack/pull/2486#issuecomment-374216201
 
 
   Packaging result: ✔centos6 ✔centos7 ✔debian. JID-1790


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Change disk offering when volume is migrated to different type of storage 
> pool.
> ---
>
> Key: CLOUDSTACK-10323
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10323
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.12
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> This is a continuation of work developed on PR #2425 (CLOUDSTACK-10240), 
> which provided root admins an override mechanism to move volumes between 
> storage systems types (local/shared) even when the disk offering would not 
> allow such operation. To complete the work, we will now provide a way for 
> administrators to enter a new disk offering that can reflect the new 
> placement of the volume. We will add an extra parameter to allow the root 
> admin inform a new disk offering for the volume. Therefore, when the volume 
> is being migrated, it will be possible to replace the disk offering to 
> reflect the new placement of the volume.
> The API method will have the following parameters: 
> * storageid (required)
> * volumeid (required)
> * livemigrate(optional)
> * newdiskofferingid (optional) – this is the new parameter
> The expected behavior is the following: 
> * If “newdiskofferingid” is not provided the current behavior is maintained. 
> Override mechanism will also keep working as we have seen so far. 
> * If the “newdiskofferingid” is provided by the admin, we will execute the 
> following checks
> ** new disk offering mode (local/shared) must match the target storage mode. 
> If it does not match, an exception will be thrown and the operator will 
> receive a message indicating the problem.
> ** we will check if the new disk offering tags match the target storage tags. 
> If it does not match, an exception will be thrown and the operator will 
> receive a message indicating the problem.
> ** check if the target storage has the capacity for the new volume. If it 
> does not have enough space, then an exception is thrown and the operator will 
> receive a message indicating the problem.
> ** check if the size of the volume is the same as the size of the new disk 
> offering. If it is not the same, we will ALLOW the change of the service 
> offering, and a warning message will be logged.
> We execute the change of the Disk offering as soon as the migration of the 
> volume finishes. Therefore, if an error happens during the migration and the 
> volume remains in the original storage system, the disk offering will keep 
> reflecting this situation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (CLOUDSTACK-10327) SSO fails with error "Session Expired", except for root admin

[Olivier Lemasle (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Olivier Lemasle updated CLOUDSTACK-10327:
-
Description: 
CloudStack SSO (using {{security.singlesignon.key}}) does not work anymore with 
CloudStack 4.11, since commit 
[9988c26|https://github.com/apache/cloudstack/commit/9988c269b259b84c0b8436bad17f88dbc1d706e7#diff-16f2bfa56c6e8760760dd2b27b47d5b4]

This commit introduced a new feature (the ability to limit admin API calls to a 
network CIDR), but also a regression due to a refactoring: every API request 
that is not "validated" generates the same error (401 - Unauthorized) and 
*invalidates the session*.

However, during an SSO login, CloudStack executes (since ACS 4.7), a [call to 
"listConfigurations"|https://github.com/apache/cloudstack/blob/8a3943b7632eddf3856a19e7d9a3fee82dd325be/ui/scripts/cloudStack.js#L172],
 an API command reserved for root admins. When the user is not a root admin, he 
does not have the privileges for this command.

With CloudStack up to 4.10, an error 432 was returned (and ignored):

{noformat}
{"errorresponse":{"uuidList":[],"errorcode":432,"cserrorcode":,"errortext":"The
 user is not allowed to request the API command or the API command does not 
exist"}}
{noformat}


With CloudStack 4.11, the error 432 is replaced by an error 401 and the session 
is invalidated. Then the next API calls lead to an error "Session Expired" and 
the user cannot log in.

{noformat}
{"listconfigurationsresponse":{"uuidList":[],"errorcode":401,"errortext":"unable
 to verify user credentials and/or request signature"}}
{noformat}


  was:
CloudStack SSO (using {{security.singlesignon.key}}) does not work anymore with 
CloudStack 4.11, since commit 
[9988c26|https://github.com/apache/cloudstack/commit/9988c269b259b84c0b8436bad17f88dbc1d706e7#diff-16f2bfa56c6e8760760dd2b27b47d5b4]

This commit introduced a new feature (the ability to limit admin API calls to a 
network CIDR), but also a regression due to a refactoring: every API request 
that is not "validated" generates the same error (401 - Unauthorized) and 
*invalidates the session*.

However, during an SSO login, CloudStack executes (since CS 4.7), a [call to 
"listConfigurations"|https://github.com/apache/cloudstack/blob/8a3943b7632eddf3856a19e7d9a3fee82dd325be/ui/scripts/cloudStack.js#L172],
 an API command reserved for root admins. When the user is not a root admin, he 
does not have the privileges for this command.

With CloudStack up to 4.10, an error 432 was returned (and ignored):

{noformat}
{"errorresponse":\{"uuidList":[],"errorcode":432,"cserrorcode":,"errortext":"The
 user is not allowed to request the API command or the API command does not 
exist"}}
{noformat}


With CloudStack 4.11, the error 432 is replaced by an error 401 and the session 
is invalidated. Then the next API calls lead to an error "Session Expired" and 
the user cannot log in.

{noformat}
{"listconfigurationsresponse":\{"uuidList":[],"errorcode":401,"errortext":"unable
 to verify user credentials and/or request signature"}}
{noformat}



> SSO fails with error "Session Expired", except for root admin
> -
>
> Key: CLOUDSTACK-10327
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10327
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: API
>Affects Versions: 4.11.0.0
>Reporter: Olivier Lemasle
>Assignee: Olivier Lemasle
>Priority: Critical
>
> CloudStack SSO (using {{security.singlesignon.key}}) does not work anymore 
> with CloudStack 4.11, since commit 
> [9988c26|https://github.com/apache/cloudstack/commit/9988c269b259b84c0b8436bad17f88dbc1d706e7#diff-16f2bfa56c6e8760760dd2b27b47d5b4]
> This commit introduced a new feature (the ability to limit admin API calls to 
> a network CIDR), but also a regression due to a refactoring: every API 
> request that is not "validated" generates the same error (401 - Unauthorized) 
> and *invalidates the session*.
> However, during an SSO login, CloudStack executes (since ACS 4.7), a [call to 
> "listConfigurations"|https://github.com/apache/cloudstack/blob/8a3943b7632eddf3856a19e7d9a3fee82dd325be/ui/scripts/cloudStack.js#L172],
>  an API command reserved for root admins. When the user is not a root admin, 
> he does not have the privileges for this command.
> With CloudStack up to 4.10, an error 432 was returned (and ignored):
> {noformat}
> {"errorresponse":{"uuidList":[],"errorcode":432,"cserrorcode":,"errortext":"The
>  user is not allowed to request the API command or the API command does not 
> exist"}}
> {noformat}
> With CloudStack 4.11, the error 432 is replaced by an error 401 and the 
> session is invalidated. Then the next API 

[jira] [Commented] (CLOUDSTACK-10214) Unable to remove local primary storage

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404726#comment-16404726
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10214:
-

blueorangutan commented on issue #2390: [CLOUDSTACK-10214] Unable to remove 
local primary storage
URL: https://github.com/apache/cloudstack/pull/2390#issuecomment-374194975
 
 
   @rafaelweingartner a Jenkins job has been kicked to build packages. I'll 
keep you posted as I make progress.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Unable to remove local primary storage 
> ---
>
> Key: CLOUDSTACK-10214
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10214
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.10.0.0, 4.9.3.0
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> When enabling the use of local storage ACS will automatically load all local 
> storage configured in the Host and start using them as primary storage to 
> deploy user VMs (if the service offering allows to do so). However, if the 
> operator wants to remove the local storage ACS will throw an exception saying 
> that the removal of local storage is not allowed.Therefore, if one wants to 
> remove a local storage, he/she needs to do a manual intervention in the 
> database and hosts.
> This limitation was removed, as it was only a logical restriction.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10226) CloudStack is not importing Local storage properly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404725#comment-16404725
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10226:
-

blueorangutan commented on issue #2401: [CLOUDSTACK-10226] CloudStack is not 
importing Local storage properly
URL: https://github.com/apache/cloudstack/pull/2401#issuecomment-374194968
 
 
   @rafaelweingartner a Jenkins job has been kicked to build packages. I'll 
keep you posted as I make progress.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> CloudStack is not importing Local storage properly 
> ---
>
> Key: CLOUDSTACK-10226
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10226
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: XenServer
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
> Fix For: 4.12
>
>
> CloudStack is importing as Local storage any XenServer SR that is of type LVM 
> or EXT. This causes a problem when one wants to use both Direct attach 
> storage and local storage. Moreover, CloudStack was not importing all of the 
> local storage that a host has available when local storage is enabled. It was 
> only importing the First SR it sees.
> To fix the first problem we started ignoring SRs that have the flag 
> shared=true when discovering local storages. SRs configured to be shared are 
> used as direct attached storage, and therefore should not be imported again 
> as local ones.
> To fix the second problem, we started loading all Local storage and importing 
> them accordingly to ACS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10226) CloudStack is not importing Local storage properly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404723#comment-16404723
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10226:
-

rafaelweingartner commented on issue #2401: [CLOUDSTACK-10226] CloudStack is 
not importing Local storage properly
URL: https://github.com/apache/cloudstack/pull/2401#issuecomment-374194761
 
 
   @blueorangutan package


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> CloudStack is not importing Local storage properly 
> ---
>
> Key: CLOUDSTACK-10226
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10226
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: XenServer
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
> Fix For: 4.12
>
>
> CloudStack is importing as Local storage any XenServer SR that is of type LVM 
> or EXT. This causes a problem when one wants to use both Direct attach 
> storage and local storage. Moreover, CloudStack was not importing all of the 
> local storage that a host has available when local storage is enabled. It was 
> only importing the First SR it sees.
> To fix the first problem we started ignoring SRs that have the flag 
> shared=true when discovering local storages. SRs configured to be shared are 
> used as direct attached storage, and therefore should not be imported again 
> as local ones.
> To fix the second problem, we started loading all Local storage and importing 
> them accordingly to ACS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10214) Unable to remove local primary storage

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404722#comment-16404722
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10214:
-

rafaelweingartner commented on issue #2390: [CLOUDSTACK-10214] Unable to remove 
local primary storage
URL: https://github.com/apache/cloudstack/pull/2390#issuecomment-374194726
 
 
   @blueorangutan package


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Unable to remove local primary storage 
> ---
>
> Key: CLOUDSTACK-10214
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10214
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.10.0.0, 4.9.3.0
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> When enabling the use of local storage ACS will automatically load all local 
> storage configured in the Host and start using them as primary storage to 
> deploy user VMs (if the service offering allows to do so). However, if the 
> operator wants to remove the local storage ACS will throw an exception saying 
> that the removal of local storage is not allowed.Therefore, if one wants to 
> remove a local storage, he/she needs to do a manual intervention in the 
> database and hosts.
> This limitation was removed, as it was only a logical restriction.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10230) User is able to change to “Guest OS type” that has been removed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404720#comment-16404720
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10230:
-

rafaelweingartner commented on issue #2404: [CLOUDSTACK-10230] User should not 
be able to use removed “Guest OS type”
URL: https://github.com/apache/cloudstack/pull/2404#issuecomment-374194653
 
 
   @blueorangutan package


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> User is able to change to “Guest OS type” that has been removed 
> 
>
> Key: CLOUDSTACK-10230
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10230
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Critical
>
> Users are able to change the OS type of VMs to “Guest OS type” that has been 
> removed. This becomes a security issue when we try to force users to use HVM 
> VMs (Meltdown/Spectre thing). A removed “guest os type” should not be usable 
> by any users in the cloud.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10230) User is able to change to “Guest OS type” that has been removed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404721#comment-16404721
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10230:
-

blueorangutan commented on issue #2404: [CLOUDSTACK-10230] User should not be 
able to use removed “Guest OS type”
URL: https://github.com/apache/cloudstack/pull/2404#issuecomment-374194714
 
 
   @rafaelweingartner a Jenkins job has been kicked to build packages. I'll 
keep you posted as I make progress.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> User is able to change to “Guest OS type” that has been removed 
> 
>
> Key: CLOUDSTACK-10230
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10230
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Critical
>
> Users are able to change the OS type of VMs to “Guest OS type” that has been 
> removed. This becomes a security issue when we try to force users to use HVM 
> VMs (Meltdown/Spectre thing). A removed “guest os type” should not be usable 
> by any users in the cloud.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10323) Change disk offering when volume is migrated to different type of storage pool.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404717#comment-16404717
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10323:
-

rafaelweingartner commented on issue #2486: [CLOUDSTACK-10323] Allow changing 
disk offering during volume migration 
URL: https://github.com/apache/cloudstack/pull/2486#issuecomment-374194408
 
 
   @blueorangutan package


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Change disk offering when volume is migrated to different type of storage 
> pool.
> ---
>
> Key: CLOUDSTACK-10323
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10323
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.12
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> This is a continuation of work developed on PR #2425 (CLOUDSTACK-10240), 
> which provided root admins an override mechanism to move volumes between 
> storage systems types (local/shared) even when the disk offering would not 
> allow such operation. To complete the work, we will now provide a way for 
> administrators to enter a new disk offering that can reflect the new 
> placement of the volume. We will add an extra parameter to allow the root 
> admin inform a new disk offering for the volume. Therefore, when the volume 
> is being migrated, it will be possible to replace the disk offering to 
> reflect the new placement of the volume.
> The API method will have the following parameters: 
> * storageid (required)
> * volumeid (required)
> * livemigrate(optional)
> * newdiskofferingid (optional) – this is the new parameter
> The expected behavior is the following: 
> * If “newdiskofferingid” is not provided the current behavior is maintained. 
> Override mechanism will also keep working as we have seen so far. 
> * If the “newdiskofferingid” is provided by the admin, we will execute the 
> following checks
> ** new disk offering mode (local/shared) must match the target storage mode. 
> If it does not match, an exception will be thrown and the operator will 
> receive a message indicating the problem.
> ** we will check if the new disk offering tags match the target storage tags. 
> If it does not match, an exception will be thrown and the operator will 
> receive a message indicating the problem.
> ** check if the target storage has the capacity for the new volume. If it 
> does not have enough space, then an exception is thrown and the operator will 
> receive a message indicating the problem.
> ** check if the size of the volume is the same as the size of the new disk 
> offering. If it is not the same, we will ALLOW the change of the service 
> offering, and a warning message will be logged.
> We execute the change of the Disk offering as soon as the migration of the 
> volume finishes. Therefore, if an error happens during the migration and the 
> volume remains in the original storage system, the disk offering will keep 
> reflecting this situation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10323) Change disk offering when volume is migrated to different type of storage pool.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404718#comment-16404718
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10323:
-

blueorangutan commented on issue #2486: [CLOUDSTACK-10323] Allow changing disk 
offering during volume migration 
URL: https://github.com/apache/cloudstack/pull/2486#issuecomment-374194467
 
 
   @rafaelweingartner a Jenkins job has been kicked to build packages. I'll 
keep you posted as I make progress.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Change disk offering when volume is migrated to different type of storage 
> pool.
> ---
>
> Key: CLOUDSTACK-10323
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10323
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.12
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> This is a continuation of work developed on PR #2425 (CLOUDSTACK-10240), 
> which provided root admins an override mechanism to move volumes between 
> storage systems types (local/shared) even when the disk offering would not 
> allow such operation. To complete the work, we will now provide a way for 
> administrators to enter a new disk offering that can reflect the new 
> placement of the volume. We will add an extra parameter to allow the root 
> admin inform a new disk offering for the volume. Therefore, when the volume 
> is being migrated, it will be possible to replace the disk offering to 
> reflect the new placement of the volume.
> The API method will have the following parameters: 
> * storageid (required)
> * volumeid (required)
> * livemigrate(optional)
> * newdiskofferingid (optional) – this is the new parameter
> The expected behavior is the following: 
> * If “newdiskofferingid” is not provided the current behavior is maintained. 
> Override mechanism will also keep working as we have seen so far. 
> * If the “newdiskofferingid” is provided by the admin, we will execute the 
> following checks
> ** new disk offering mode (local/shared) must match the target storage mode. 
> If it does not match, an exception will be thrown and the operator will 
> receive a message indicating the problem.
> ** we will check if the new disk offering tags match the target storage tags. 
> If it does not match, an exception will be thrown and the operator will 
> receive a message indicating the problem.
> ** check if the target storage has the capacity for the new volume. If it 
> does not have enough space, then an exception is thrown and the operator will 
> receive a message indicating the problem.
> ** check if the size of the volume is the same as the size of the new disk 
> offering. If it is not the same, we will ALLOW the change of the service 
> offering, and a warning message will be logged.
> We execute the change of the Disk offering as soon as the migration of the 
> volume finishes. Therefore, if an error happens during the migration and the 
> volume remains in the original storage system, the disk offering will keep 
> reflecting this situation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10278) Adding a SQL table column is not Idempotent

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404699#comment-16404699
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10278:
-

ernjvr commented on issue #2449: WIP CLOUDSTACK-10278 idempotent column addition
URL: https://github.com/apache/cloudstack/pull/2449#issuecomment-374191891
 
 
   @marcaurele using these stored procedures adds functionality independent of 
any database version. I did consider the IF NOT EXISTS option. According to the 
link you supplied it is only available from version 10.3.0 but the according to 
the installation guide "CloudStack has been tested with MySQL 5.1 and 5.5. 
These versions are included in RHEL/CentOS and Ubuntu." The stored procedures 
also address foreign keys and indexes.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Adding a SQL table column is not Idempotent
> ---
>
> Key: CLOUDSTACK-10278
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10278
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Install and Setup
>Affects Versions: 4.10.0.0, 4.11.0.0
>Reporter: Ernie Janse van Rensburg
>Assignee: Ernie Janse van Rensburg
>Priority: Major
>   Original Estimate: 4h
>  Remaining Estimate: 4h
>
> The SQL code to add a new column to a table in the 
> META-INF/db/schema-41000to41100.sql script is not written in an idempotent 
> way. When the upgrade is re-run, the code above causes a SQL error as 
> reported on the user mailing list: 
> ERROR [c.c.u.d.ScriptRunner] (main:null) (logid:)
> Error executing: ALTER TABLE cloud.network_offerings ADD COLUMN for_vpc
> INT(1) NOT NULL DEFAULT 0
> This is a more generic problem for every version due to to the fact that it 
> is not idempotent
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10323) Change disk offering when volume is migrated to different type of storage pool.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404622#comment-16404622
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10323:
-

nitin-maharana commented on a change in pull request #2486: [CLOUDSTACK-10323] 
Allow changing disk offering during volume migration 
URL: https://github.com/apache/cloudstack/pull/2486#discussion_r175391456
 
 

 ##
 File path: server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java
 ##
 @@ -2097,26 +2079,102 @@ public Volume migrateVolume(MigrateVolumeCmd cmd) {
 }
 }
 
-return orchestrateMigrateVolume(vol.getId(), destPool.getId(), 
liveMigrateVolume);
+return orchestrateMigrateVolume(vol, destPool, liveMigrateVolume, 
newDiskOffering);
 }
 
-private Volume orchestrateMigrateVolume(long volumeId, long destPoolId, 
boolean liveMigrateVolume) {
-VolumeVO vol = _volsDao.findById(volumeId);
-assert (vol != null);
-StoragePool destPool = 
(StoragePool)dataStoreMgr.getDataStore(destPoolId, DataStoreRole.Primary);
-assert (destPool != null);
+/**
+ * Retrieve the new disk offering UUID that might be sent to replace the 
current one in the volume being migrated.
+ * If no disk offering UUID is provided we return null. Otherwise, we 
perform the following checks.
+ * 
 
 Review comment:
   Got it. Thanks.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Change disk offering when volume is migrated to different type of storage 
> pool.
> ---
>
> Key: CLOUDSTACK-10323
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10323
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.12
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> This is a continuation of work developed on PR #2425 (CLOUDSTACK-10240), 
> which provided root admins an override mechanism to move volumes between 
> storage systems types (local/shared) even when the disk offering would not 
> allow such operation. To complete the work, we will now provide a way for 
> administrators to enter a new disk offering that can reflect the new 
> placement of the volume. We will add an extra parameter to allow the root 
> admin inform a new disk offering for the volume. Therefore, when the volume 
> is being migrated, it will be possible to replace the disk offering to 
> reflect the new placement of the volume.
> The API method will have the following parameters: 
> * storageid (required)
> * volumeid (required)
> * livemigrate(optional)
> * newdiskofferingid (optional) – this is the new parameter
> The expected behavior is the following: 
> * If “newdiskofferingid” is not provided the current behavior is maintained. 
> Override mechanism will also keep working as we have seen so far. 
> * If the “newdiskofferingid” is provided by the admin, we will execute the 
> following checks
> ** new disk offering mode (local/shared) must match the target storage mode. 
> If it does not match, an exception will be thrown and the operator will 
> receive a message indicating the problem.
> ** we will check if the new disk offering tags match the target storage tags. 
> If it does not match, an exception will be thrown and the operator will 
> receive a message indicating the problem.
> ** check if the target storage has the capacity for the new volume. If it 
> does not have enough space, then an exception is thrown and the operator will 
> receive a message indicating the problem.
> ** check if the size of the volume is the same as the size of the new disk 
> offering. If it is not the same, we will ALLOW the change of the service 
> offering, and a warning message will be logged.
> We execute the change of the Disk offering as soon as the migration of the 
> volume finishes. Therefore, if an error happens during the migration and the 
> volume remains in the original storage system, the disk offering will keep 
> reflecting this situation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10323) Change disk offering when volume is migrated to different type of storage pool.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404619#comment-16404619
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10323:
-

rafaelweingartner commented on a change in pull request #2486: 
[CLOUDSTACK-10323] Allow changing disk offering during volume migration 
URL: https://github.com/apache/cloudstack/pull/2486#discussion_r175390104
 
 

 ##
 File path: server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java
 ##
 @@ -2097,26 +2079,102 @@ public Volume migrateVolume(MigrateVolumeCmd cmd) {
 }
 }
 
-return orchestrateMigrateVolume(vol.getId(), destPool.getId(), 
liveMigrateVolume);
+return orchestrateMigrateVolume(vol, destPool, liveMigrateVolume, 
newDiskOffering);
 }
 
-private Volume orchestrateMigrateVolume(long volumeId, long destPoolId, 
boolean liveMigrateVolume) {
-VolumeVO vol = _volsDao.findById(volumeId);
-assert (vol != null);
-StoragePool destPool = 
(StoragePool)dataStoreMgr.getDataStore(destPoolId, DataStoreRole.Primary);
-assert (destPool != null);
+/**
+ * Retrieve the new disk offering UUID that might be sent to replace the 
current one in the volume being migrated.
+ * If no disk offering UUID is provided we return null. Otherwise, we 
perform the following checks.
+ * 
 
 Review comment:
   It is not by mistake. It is on purpose to format the java documentation. 
Using them you are able to create javadocs as this one:
   
![javadoc](https://user-images.githubusercontent.com/4129005/37590952-a3137f44-2b48-11e8-9408-a32906dc0732.png)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Change disk offering when volume is migrated to different type of storage 
> pool.
> ---
>
> Key: CLOUDSTACK-10323
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10323
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.12
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> This is a continuation of work developed on PR #2425 (CLOUDSTACK-10240), 
> which provided root admins an override mechanism to move volumes between 
> storage systems types (local/shared) even when the disk offering would not 
> allow such operation. To complete the work, we will now provide a way for 
> administrators to enter a new disk offering that can reflect the new 
> placement of the volume. We will add an extra parameter to allow the root 
> admin inform a new disk offering for the volume. Therefore, when the volume 
> is being migrated, it will be possible to replace the disk offering to 
> reflect the new placement of the volume.
> The API method will have the following parameters: 
> * storageid (required)
> * volumeid (required)
> * livemigrate(optional)
> * newdiskofferingid (optional) – this is the new parameter
> The expected behavior is the following: 
> * If “newdiskofferingid” is not provided the current behavior is maintained. 
> Override mechanism will also keep working as we have seen so far. 
> * If the “newdiskofferingid” is provided by the admin, we will execute the 
> following checks
> ** new disk offering mode (local/shared) must match the target storage mode. 
> If it does not match, an exception will be thrown and the operator will 
> receive a message indicating the problem.
> ** we will check if the new disk offering tags match the target storage tags. 
> If it does not match, an exception will be thrown and the operator will 
> receive a message indicating the problem.
> ** check if the target storage has the capacity for the new volume. If it 
> does not have enough space, then an exception is thrown and the operator will 
> receive a message indicating the problem.
> ** check if the size of the volume is the same as the size of the new disk 
> offering. If it is not the same, we will ALLOW the change of the service 
> offering, and a warning message will be logged.
> We execute the change of the Disk offering as soon as the migration of the 
> volume finishes. Therefore, if an error happens during the migration and the 
> volume remains in the original storage system, the disk offering will keep 
> reflecting this situation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10323) Change disk offering when volume is migrated to different type of storage pool.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404616#comment-16404616
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10323:
-

nitin-maharana commented on a change in pull request #2486: [CLOUDSTACK-10323] 
Allow changing disk offering during volume migration 
URL: https://github.com/apache/cloudstack/pull/2486#discussion_r175389207
 
 

 ##
 File path: server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java
 ##
 @@ -2097,26 +2079,102 @@ public Volume migrateVolume(MigrateVolumeCmd cmd) {
 }
 }
 
-return orchestrateMigrateVolume(vol.getId(), destPool.getId(), 
liveMigrateVolume);
+return orchestrateMigrateVolume(vol, destPool, liveMigrateVolume, 
newDiskOffering);
 }
 
-private Volume orchestrateMigrateVolume(long volumeId, long destPoolId, 
boolean liveMigrateVolume) {
-VolumeVO vol = _volsDao.findById(volumeId);
-assert (vol != null);
-StoragePool destPool = 
(StoragePool)dataStoreMgr.getDataStore(destPoolId, DataStoreRole.Primary);
-assert (destPool != null);
+/**
+ * Retrieve the new disk offering UUID that might be sent to replace the 
current one in the volume being migrated.
+ * If no disk offering UUID is provided we return null. Otherwise, we 
perform the following checks.
+ * 
 
 Review comment:
   @rafaelweingartner, I think you have added few HTML tags by mistake here. 
Not sure if it was intended.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Change disk offering when volume is migrated to different type of storage 
> pool.
> ---
>
> Key: CLOUDSTACK-10323
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10323
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.12
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> This is a continuation of work developed on PR #2425 (CLOUDSTACK-10240), 
> which provided root admins an override mechanism to move volumes between 
> storage systems types (local/shared) even when the disk offering would not 
> allow such operation. To complete the work, we will now provide a way for 
> administrators to enter a new disk offering that can reflect the new 
> placement of the volume. We will add an extra parameter to allow the root 
> admin inform a new disk offering for the volume. Therefore, when the volume 
> is being migrated, it will be possible to replace the disk offering to 
> reflect the new placement of the volume.
> The API method will have the following parameters: 
> * storageid (required)
> * volumeid (required)
> * livemigrate(optional)
> * newdiskofferingid (optional) – this is the new parameter
> The expected behavior is the following: 
> * If “newdiskofferingid” is not provided the current behavior is maintained. 
> Override mechanism will also keep working as we have seen so far. 
> * If the “newdiskofferingid” is provided by the admin, we will execute the 
> following checks
> ** new disk offering mode (local/shared) must match the target storage mode. 
> If it does not match, an exception will be thrown and the operator will 
> receive a message indicating the problem.
> ** we will check if the new disk offering tags match the target storage tags. 
> If it does not match, an exception will be thrown and the operator will 
> receive a message indicating the problem.
> ** check if the target storage has the capacity for the new volume. If it 
> does not have enough space, then an exception is thrown and the operator will 
> receive a message indicating the problem.
> ** check if the size of the volume is the same as the size of the new disk 
> offering. If it is not the same, we will ALLOW the change of the service 
> offering, and a warning message will be logged.
> We execute the change of the Disk offering as soon as the migration of the 
> volume finishes. Therefore, if an error happens during the migration and the 
> volume remains in the original storage system, the disk offering will keep 
> reflecting this situation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10323) Change disk offering when volume is migrated to different type of storage pool.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404611#comment-16404611
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10323:
-

nitin-maharana commented on a change in pull request #2486: [CLOUDSTACK-10323] 
Allow changing disk offering during volume migration 
URL: https://github.com/apache/cloudstack/pull/2486#discussion_r175389207
 
 

 ##
 File path: server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java
 ##
 @@ -2097,26 +2079,102 @@ public Volume migrateVolume(MigrateVolumeCmd cmd) {
 }
 }
 
-return orchestrateMigrateVolume(vol.getId(), destPool.getId(), 
liveMigrateVolume);
+return orchestrateMigrateVolume(vol, destPool, liveMigrateVolume, 
newDiskOffering);
 }
 
-private Volume orchestrateMigrateVolume(long volumeId, long destPoolId, 
boolean liveMigrateVolume) {
-VolumeVO vol = _volsDao.findById(volumeId);
-assert (vol != null);
-StoragePool destPool = 
(StoragePool)dataStoreMgr.getDataStore(destPoolId, DataStoreRole.Primary);
-assert (destPool != null);
+/**
+ * Retrieve the new disk offering UUID that might be sent to replace the 
current one in the volume being migrated.
+ * If no disk offering UUID is provided we return null. Otherwise, we 
perform the following checks.
+ * 
 
 Review comment:
   @rafaelweingartner, I think you have added few HTML tags by mistake here.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Change disk offering when volume is migrated to different type of storage 
> pool.
> ---
>
> Key: CLOUDSTACK-10323
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10323
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.12
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> This is a continuation of work developed on PR #2425 (CLOUDSTACK-10240), 
> which provided root admins an override mechanism to move volumes between 
> storage systems types (local/shared) even when the disk offering would not 
> allow such operation. To complete the work, we will now provide a way for 
> administrators to enter a new disk offering that can reflect the new 
> placement of the volume. We will add an extra parameter to allow the root 
> admin inform a new disk offering for the volume. Therefore, when the volume 
> is being migrated, it will be possible to replace the disk offering to 
> reflect the new placement of the volume.
> The API method will have the following parameters: 
> * storageid (required)
> * volumeid (required)
> * livemigrate(optional)
> * newdiskofferingid (optional) – this is the new parameter
> The expected behavior is the following: 
> * If “newdiskofferingid” is not provided the current behavior is maintained. 
> Override mechanism will also keep working as we have seen so far. 
> * If the “newdiskofferingid” is provided by the admin, we will execute the 
> following checks
> ** new disk offering mode (local/shared) must match the target storage mode. 
> If it does not match, an exception will be thrown and the operator will 
> receive a message indicating the problem.
> ** we will check if the new disk offering tags match the target storage tags. 
> If it does not match, an exception will be thrown and the operator will 
> receive a message indicating the problem.
> ** check if the target storage has the capacity for the new volume. If it 
> does not have enough space, then an exception is thrown and the operator will 
> receive a message indicating the problem.
> ** check if the size of the volume is the same as the size of the new disk 
> offering. If it is not the same, we will ALLOW the change of the service 
> offering, and a warning message will be logged.
> We execute the change of the Disk offering as soon as the migration of the 
> volume finishes. Therefore, if an error happens during the migration and the 
> volume remains in the original storage system, the disk offering will keep 
> reflecting this situation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10323) Change disk offering when volume is migrated to different type of storage pool.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404612#comment-16404612
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10323:
-

nitin-maharana commented on a change in pull request #2486: [CLOUDSTACK-10323] 
Allow changing disk offering during volume migration 
URL: https://github.com/apache/cloudstack/pull/2486#discussion_r175389327
 
 

 ##
 File path: server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java
 ##
 @@ -2097,26 +2079,102 @@ public Volume migrateVolume(MigrateVolumeCmd cmd) {
 }
 }
 
-return orchestrateMigrateVolume(vol.getId(), destPool.getId(), 
liveMigrateVolume);
+return orchestrateMigrateVolume(vol, destPool, liveMigrateVolume, 
newDiskOffering);
 }
 
-private Volume orchestrateMigrateVolume(long volumeId, long destPoolId, 
boolean liveMigrateVolume) {
-VolumeVO vol = _volsDao.findById(volumeId);
-assert (vol != null);
-StoragePool destPool = 
(StoragePool)dataStoreMgr.getDataStore(destPoolId, DataStoreRole.Primary);
-assert (destPool != null);
+/**
+ * Retrieve the new disk offering UUID that might be sent to replace the 
current one in the volume being migrated.
+ * If no disk offering UUID is provided we return null. Otherwise, we 
perform the following checks.
+ * 
+ *  Is the disk offering UUID entered valid? If not, an  {@link 
InvalidParameterValueException} is thrown;
+ *  If the disk offering was already removed, we thrown an {@link 
InvalidParameterValueException} is thrown;
+ *  We then check if the user executing the operation has access to 
the given disk offering.
+ * 
+ *
+ * If all checks pass, we move forward returning the disk offering object.
+ */
+private DiskOfferingVO retrieveAndValidateNewDiskOffering(MigrateVolumeCmd 
cmd) {
+String newDiskOfferingUuid = cmd.getNewDiskOfferingUuid();
+if (org.apache.commons.lang.StringUtils.isBlank(newDiskOfferingUuid)) {
+return null;
+}
+DiskOfferingVO newDiskOffering = 
_diskOfferingDao.findByUuid(newDiskOfferingUuid);
+if (newDiskOffering == null) {
+throw new InvalidParameterValueException(String.format("The disk 
offering informed is not valid [id=%s].", newDiskOfferingUuid));
+}
+if (newDiskOffering.getRemoved() != null) {
+throw new InvalidParameterValueException(String.format("We cannot 
assign a removed disk offering [id=%s] to a volume. ", 
newDiskOffering.getUuid()));
+}
+Account caller = CallContext.current().getCallingAccount();
+_accountMgr.checkAccess(caller, newDiskOffering);
+return newDiskOffering;
+}
+
+/**
+ * Performs the validations required for replacing the disk offering while 
migrating the volume of storage. If no new disk offering is provided, we do not 
execute any validation.
+ * If a disk offering is informed, we then proceed with the following 
checks.
+ * 
 
 Review comment:
   Same here.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Change disk offering when volume is migrated to different type of storage 
> pool.
> ---
>
> Key: CLOUDSTACK-10323
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10323
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.12
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> This is a continuation of work developed on PR #2425 (CLOUDSTACK-10240), 
> which provided root admins an override mechanism to move volumes between 
> storage systems types (local/shared) even when the disk offering would not 
> allow such operation. To complete the work, we will now provide a way for 
> administrators to enter a new disk offering that can reflect the new 
> placement of the volume. We will add an extra parameter to allow the root 
> admin inform a new disk offering for the volume. Therefore, when the volume 
> is being migrated, it will be possible to replace the disk offering to 
> reflect the new placement of the volume.
> The API method will have the following parameters: 
> * storageid (required)
> * volumeid (required)
> * livemigrate(optional)
> * newdiskofferingid (optional) – this is the new parameter
> The expected behavior is the following: 
> * If “newdiskofferingid” is not provided 

[jira] [Commented] (CLOUDSTACK-10301) Allow updating the network ACL list name and Description

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404593#comment-16404593
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10301:
-

rafaelweingartner commented on issue #2462: [CLOUDSTACK-10301] Allow updating 
the network ACL list name and Description
URL: https://github.com/apache/cloudstack/pull/2462#issuecomment-374155496
 
 
   @DaanHoogland, this PR is not for a comment/description in ACL. This is to 
allow editing ACL listing description and name. Once you create an ACL listing, 
right now, you are not allowed to edit the name and description. This is quite 
annoying, if you have a typo or some incomplete sentence you cannot change 
them. You would need to remove and add the ACL again (or you can go directly in 
the database like I have done already so many times).
   
   I attached an screenshot to show what fields I am allowing updating with 
this PR.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Allow updating the network ACL list name and Description 
> -
>
> Key: CLOUDSTACK-10301
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10301
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> Currently, it is not possible to update the Network ACL name or Description 
> of an ACL after it has been created. It might be interesting to expose this 
> option via API and to add this feature to the UI.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10326) Prevent hosts fall into Maintenance when there are running VMs on it

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404569#comment-16404569
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10326:
-

nitin-maharana commented on a change in pull request #2493: CLOUDSTACK-10326: 
Prevent hosts fall into Maintenance when there are running VMs on it
URL: https://github.com/apache/cloudstack/pull/2493#discussion_r175377422
 
 

 ##
 File path: server/src/com/cloud/resource/ResourceManagerImpl.java
 ##
 @@ -1296,10 +1296,17 @@ public boolean checkAndMaintain(final long hostId) {
 if (host.getType() != Host.Type.Storage) {
 final List vos = _vmDao.listByHostId(hostId);
 final List vosMigrating = 
_vmDao.listVmsMigratingFromHost(hostId);
+final List failedMigratedVms = 
_vmDao.listNonMigratingVmsByHostEqualsLastHost(hostId);
 if (vos.isEmpty() && vosMigrating.isEmpty()) {
-resourceStateTransitTo(host, 
ResourceState.Event.InternalEnterMaintenance, _nodeId);
-hostInMaintenance = true;
-
ActionEventUtils.onCompletedActionEvent(CallContext.current().getCallingUserId(),
 CallContext.current().getCallingAccountId(), EventVO.LEVEL_INFO, 
EventTypes.EVENT_MAINTENANCE_PREPARE, "completed maintenance for host " + 
hostId, 0);
+if (!failedMigratedVms.isEmpty()) {
 
 Review comment:
   @nvazquez, I think if we put this condition in line 1300 itself, then it 
would maintain the old flow. Something like this in line 1300: if 
(vos.isEmpty() && vosMigrating.isEmpty() && failedMigratedVms.isEmpty()) {...}. 
And we can put this message(debug purpose) for all kind of failures, where 
resource state would transit to UnableToMigrate.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Prevent hosts fall into Maintenance when there are running VMs on it
> 
>
> Key: CLOUDSTACK-10326
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10326
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.11.0.0
>Reporter: Nicolas Vazquez
>Assignee: Nicolas Vazquez
>Priority: Major
> Fix For: 4.11.1.0
>
> Attachments: CLOUDSTACK-10326-Debug.png, 
> CLOUDSTACK-10326-InitialState.png, CLOUDSTACK-10326-Migrating.png, 
> CLOUDSTACK-10326-MigrationFailed.png
>
>
> This issue was discovered, fixed and tested on KVM, but applies for every 
> hypervisor.
> h2. Background
> When enabling maintenance mode in a host, host state is put into 
> 'PrepareForMaintenance' and running VMs are migrated into another host. After 
> every VM is migrated, host goes to 'Maintenance' state.
> Checks are performed on ResourceManagerImpl.checkAndMaintan() method:
>  * List VMs with host_id = HOST_ID
>  * List VMs with last_host_id = HOST_ID and state=Migrating
> When both queries are empty, then the host can be put into Maintenance.
> When a VM is being migrated to DEST_HOST, its host_id column is set to 
> DEST_HOST, last_host_id = ORIGIN_HOST and state = Migrating. If then 
> migration fails, host_id = last_host_id = ORIGIN_HOST 
> h2. Issue
> This sequence:
>  * Enable maintenance mode on ORIGIN_HOST
>  * VMs start being migrated to a host, say DEST_HOST
>  * checkAndMaintain() starts:
>  ** First check passes (no VM with host_id = ORIGIN_HOST_ID as those are 
> being migrated)
>  ** Before the second check, one or more migrations fail
>  ** Second check passes, however there are VMs running on the host as 
> migrations have failed.
>  * Host goes into Maintenance state.
> Screenshots attached, query executed on each case:
> select id, name, instance_name, state, host_id, last_host_id from vm_instance;



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10326) Prevent hosts fall into Maintenance when there are running VMs on it

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404568#comment-16404568
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10326:
-

nitin-maharana commented on a change in pull request #2493: CLOUDSTACK-10326: 
Prevent hosts fall into Maintenance when there are running VMs on it
URL: https://github.com/apache/cloudstack/pull/2493#discussion_r175377422
 
 

 ##
 File path: server/src/com/cloud/resource/ResourceManagerImpl.java
 ##
 @@ -1296,10 +1296,17 @@ public boolean checkAndMaintain(final long hostId) {
 if (host.getType() != Host.Type.Storage) {
 final List vos = _vmDao.listByHostId(hostId);
 final List vosMigrating = 
_vmDao.listVmsMigratingFromHost(hostId);
+final List failedMigratedVms = 
_vmDao.listNonMigratingVmsByHostEqualsLastHost(hostId);
 if (vos.isEmpty() && vosMigrating.isEmpty()) {
-resourceStateTransitTo(host, 
ResourceState.Event.InternalEnterMaintenance, _nodeId);
-hostInMaintenance = true;
-
ActionEventUtils.onCompletedActionEvent(CallContext.current().getCallingUserId(),
 CallContext.current().getCallingAccountId(), EventVO.LEVEL_INFO, 
EventTypes.EVENT_MAINTENANCE_PREPARE, "completed maintenance for host " + 
hostId, 0);
+if (!failedMigratedVms.isEmpty()) {
 
 Review comment:
   @nvazquez, I think if we put this condition in line 1300 itself, then it 
would maintain the old flow. Something like this in line 1300: if 
(vos.isEmpty() && vosMigrating.isEmpty() && failedMigratedVms.isEmpty()) {...}. 
And we can put this message for all kind of failures, where resource state 
would transit to UnableToMigrate.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Prevent hosts fall into Maintenance when there are running VMs on it
> 
>
> Key: CLOUDSTACK-10326
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10326
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Affects Versions: 4.11.0.0
>Reporter: Nicolas Vazquez
>Assignee: Nicolas Vazquez
>Priority: Major
> Fix For: 4.11.1.0
>
> Attachments: CLOUDSTACK-10326-Debug.png, 
> CLOUDSTACK-10326-InitialState.png, CLOUDSTACK-10326-Migrating.png, 
> CLOUDSTACK-10326-MigrationFailed.png
>
>
> This issue was discovered, fixed and tested on KVM, but applies for every 
> hypervisor.
> h2. Background
> When enabling maintenance mode in a host, host state is put into 
> 'PrepareForMaintenance' and running VMs are migrated into another host. After 
> every VM is migrated, host goes to 'Maintenance' state.
> Checks are performed on ResourceManagerImpl.checkAndMaintan() method:
>  * List VMs with host_id = HOST_ID
>  * List VMs with last_host_id = HOST_ID and state=Migrating
> When both queries are empty, then the host can be put into Maintenance.
> When a VM is being migrated to DEST_HOST, its host_id column is set to 
> DEST_HOST, last_host_id = ORIGIN_HOST and state = Migrating. If then 
> migration fails, host_id = last_host_id = ORIGIN_HOST 
> h2. Issue
> This sequence:
>  * Enable maintenance mode on ORIGIN_HOST
>  * VMs start being migrated to a host, say DEST_HOST
>  * checkAndMaintain() starts:
>  ** First check passes (no VM with host_id = ORIGIN_HOST_ID as those are 
> being migrated)
>  ** Before the second check, one or more migrations fail
>  ** Second check passes, however there are VMs running on the host as 
> migrations have failed.
>  * Host goes into Maintenance state.
> Screenshots attached, query executed on each case:
> select id, name, instance_name, state, host_id, last_host_id from vm_instance;



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10307) Remove unused things from HostDaoImpl

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404563#comment-16404563
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10307:
-

rafaelweingartner commented on issue #2438: [CLOUDSTACK-10307] Remove unused 
things from HostDaoImpl
URL: https://github.com/apache/cloudstack/pull/2438#issuecomment-374155945
 
 
   So, I wait for the results.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Remove unused things from HostDaoImpl
> -
>
> Key: CLOUDSTACK-10307
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10307
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Minor
>
> Remove unnecessary annotation of HostDaoImpl. While removing this annotation 
> I noticed that one of the methods were not necessary. While removing it, I 
> found some code in CloudZonesStartupProcessor that was also not used, and 
> removed it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10301) Allow updating the network ACL list name and Description

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404561#comment-16404561
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10301:
-

rafaelweingartner commented on issue #2462: [CLOUDSTACK-10301] Allow updating 
the network ACL list name and Description
URL: https://github.com/apache/cloudstack/pull/2462#issuecomment-374155496
 
 
   @DaanHoogland, this PR is not for a comment/description in ACL. This is to 
allow editing ACL listing description and name. Once you create an ACL listing, 
right now, you are not allowed to edit the name and description. This is quite 
annoying, if you have a typo or some incomplete sentence you cannot change 
them. You would need to remove and add the ACL again (or you can go directly in 
the database like I have done already so many times).


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Allow updating the network ACL list name and Description 
> -
>
> Key: CLOUDSTACK-10301
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10301
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> Currently, it is not possible to update the Network ACL name or Description 
> of an ACL after it has been created. It might be interesting to expose this 
> option via API and to add this feature to the UI.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10320) Invalid pair for response object breaking response parsing

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404551#comment-16404551
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10320:
-

marcaurele commented on a change in pull request #2481: CLOUDSTACK-10320 - 
Invalid pair for response object breaking response parsing
URL: https://github.com/apache/cloudstack/pull/2481#discussion_r175355297
 
 

 ##
 File path: framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java
 ##
 @@ -1323,6 +1327,11 @@ protected void addJoins(StringBuilder str, 
Collection searchAndDistinctCount(final 
SearchCriteria sc, final Filter filter) {
 List objects = search(sc, filter, null, false);
 Integer count = getDistinctCount(sc);
+// Count cannot be 0 if there is at least a result in the list, see 
CLOUDSTACK-10320
+if (count == 0 && !objects.isEmpty()) {
+// Cannot assume if it's more than one since the count is distinct 
vs search
+count = 1;
 
 Review comment:
   Because the search is mostly done against views which don't return distinct 
rows due to the *not so smart* queries fetching the ~details~ resource tags 
rows. Look at the APIHelper with iterate on duplicate row entries, but only 
append the details.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Invalid pair for response object breaking response parsing
> --
>
> Key: CLOUDSTACK-10320
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10320
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: API
>Reporter: Marc-Aurèle Brothier
>Assignee: Marc-Aurèle Brothier
>Priority: Major
>
> Under some circumstances, the API is returning an invalid response, for 
> simplicity I will expose the JSON case. The API response on a 
> listVirtualMachines can be this string:
> {code:java}
> { "listvirtualmachinesresponse" :  ] } }{code}
> To understand how this is possible, assume you have more than one management 
> server and one is processing the destroy of a virtual machine in the account 
> X which is the only one it has. Another process is returning the result of 
> listVirtualMachines for that same account X. During the listVM command, the 
> result set is fetch with a searchAndDistinctCount due to the view 
> ([https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/query/QueryManagerImpl.java#L1024).]
>  This is done through 2 queries in the GenericDao 
> [https://github.com/apache/cloudstack/blob/master/framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java#L1323]
>  and if you encounter the _right_ conditions, the VM will be marked as 
> removed in between those 2 queries. This results in having a Pair result with 
> at least one object but a count of 0. Then following how is done the 
> serialization of the response at 
> [https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/response/ApiResponseSerializer.java#L86]
>  you will reach the case where your output is the one previously mentioned.
> To overcome this issue, there isn't a true fix but only a better pair 
> response to ensure a correct response formatting. If the result set contains 
> at least something, the count cannot be 0 but we cannot guess the correct 
> answer, but only state it has at least one element.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10278) Adding a SQL table column is not Idempotent

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404536#comment-16404536
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10278:
-

marcaurele commented on issue #2449: WIP CLOUDSTACK-10278 idempotent column 
addition
URL: https://github.com/apache/cloudstack/pull/2449#issuecomment-374144380
 
 
   https://mariadb.com/kb/en/library/alter-table/#add-column


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Adding a SQL table column is not Idempotent
> ---
>
> Key: CLOUDSTACK-10278
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10278
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Install and Setup
>Affects Versions: 4.10.0.0, 4.11.0.0
>Reporter: Ernie Janse van Rensburg
>Assignee: Ernie Janse van Rensburg
>Priority: Major
>   Original Estimate: 4h
>  Remaining Estimate: 4h
>
> The SQL code to add a new column to a table in the 
> META-INF/db/schema-41000to41100.sql script is not written in an idempotent 
> way. When the upgrade is re-run, the code above causes a SQL error as 
> reported on the user mailing list: 
> ERROR [c.c.u.d.ScriptRunner] (main:null) (logid:)
> Error executing: ALTER TABLE cloud.network_offerings ADD COLUMN for_vpc
> INT(1) NOT NULL DEFAULT 0
> This is a more generic problem for every version due to to the fact that it 
> is not idempotent
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10278) Adding a SQL table column is not Idempotent

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404522#comment-16404522
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10278:
-

marcaurele commented on issue #2449: WIP CLOUDSTACK-10278 idempotent column 
addition
URL: https://github.com/apache/cloudstack/pull/2449#issuecomment-374139925
 
 
   @ernjvr what are the benefits of using such store procedure versus using the 
explicit SQL syntax `IF NOT EXISTS` / `IF EXISTS` during `ALTER TABLE` queries?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Adding a SQL table column is not Idempotent
> ---
>
> Key: CLOUDSTACK-10278
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10278
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Install and Setup
>Affects Versions: 4.10.0.0, 4.11.0.0
>Reporter: Ernie Janse van Rensburg
>Assignee: Ernie Janse van Rensburg
>Priority: Major
>   Original Estimate: 4h
>  Remaining Estimate: 4h
>
> The SQL code to add a new column to a table in the 
> META-INF/db/schema-41000to41100.sql script is not written in an idempotent 
> way. When the upgrade is re-run, the code above causes a SQL error as 
> reported on the user mailing list: 
> ERROR [c.c.u.d.ScriptRunner] (main:null) (logid:)
> Error executing: ALTER TABLE cloud.network_offerings ADD COLUMN for_vpc
> INT(1) NOT NULL DEFAULT 0
> This is a more generic problem for every version due to to the fact that it 
> is not idempotent
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10262) create an API driven System VM upgrade posibility

[Paul Angus (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404517#comment-16404517
 ] 

Paul Angus commented on CLOUDSTACK-10262:
-

Hi [~rohit.vavaldas]

To add some colour to this, here is the original brief, from which Daan took 
the headline feature.
 * As a cloud operator, I want to be able to download and install the system VM 
templates via a running CloudStack mgmt. server.  I do not want to have to 
'pre-seed' templates manually. I wish the 'add zone' wizard to take me through 
the process of specifying the location of the system VM template to install if 
i don't do it through the API.
 * I want a UI and API to upload system VM templates with or without the aid of 
the SSVM. I wish to be able to specify a ‘custom’ local (file) or remote (url) 
location of the system VM template or allow CloudStack to download the 
‘default’ template for the current hypervisor(s) installed.
 * Through the UI and API, I wish to be able to upload a new system VM template 
(separately from any upgrade).
 * I wish to be able to set/select which template(s) to use as system VM 
templates easily through the UI and API.

This may give you some ideas.

 

 

> create an API driven System VM upgrade posibility
> -
>
> Key: CLOUDSTACK-10262
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10262
> Project: CloudStack
>  Issue Type: Wish
>Reporter: Daan Hoogland
>Priority: Major
>  Labels: gsoc2018
>
> As a cloud admin I want a single call API to upgrade/upload a new System VM 
> template. The System VM template upgrade procedure is now largely manual and 
> error prone. An one ring to bind them all API call to replace the current 
> template for system VMs would help cloud admins world wide.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10301) Allow updating the network ACL list name and Description

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404512#comment-16404512
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10301:
-

DaanHoogland commented on issue #2462: [CLOUDSTACK-10301] Allow updating the 
network ACL list name and Description
URL: https://github.com/apache/cloudstack/pull/2462#issuecomment-374137976
 
 
   @rafaelweingartner user comments can be added to a list (enum) of entity 
types: HOST("host"), DOMAIN("domain"), VM("vm_instance")
   at this moment with the annotations API. addAnnotation, etc. This is 
extendable to other entities. The annotations left on entities this way leaves 
an audit trail by user id. Useful for admins. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Allow updating the network ACL list name and Description 
> -
>
> Key: CLOUDSTACK-10301
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10301
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Major
>
> Currently, it is not possible to update the Network ACL name or Description 
> of an ACL after it has been created. It might be interesting to expose this 
> option via API and to add this feature to the UI.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10307) Remove unused things from HostDaoImpl

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404499#comment-16404499
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10307:
-

blueorangutan commented on issue #2438: [CLOUDSTACK-10307] Remove unused things 
from HostDaoImpl
URL: https://github.com/apache/cloudstack/pull/2438#issuecomment-374136289
 
 
   @DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + vmware-60u2) has 
been kicked to run smoke tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Remove unused things from HostDaoImpl
> -
>
> Key: CLOUDSTACK-10307
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10307
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Minor
>
> Remove unnecessary annotation of HostDaoImpl. While removing this annotation 
> I noticed that one of the methods were not necessary. While removing it, I 
> found some code in CloudZonesStartupProcessor that was also not used, and 
> removed it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10307) Remove unused things from HostDaoImpl

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404497#comment-16404497
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10307:
-

DaanHoogland commented on issue #2438: [CLOUDSTACK-10307] Remove unused things 
from HostDaoImpl
URL: https://github.com/apache/cloudstack/pull/2438#issuecomment-374136059
 
 
   @borisstoyanov @rafaelweingartner i have not seen any succesful tests for 
vmware on this; not merging!
   @blueorangutan test centos7 vmware-60u2


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Remove unused things from HostDaoImpl
> -
>
> Key: CLOUDSTACK-10307
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10307
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Reporter: Rafael Weingärtner
>Assignee: Rafael Weingärtner
>Priority: Minor
>
> Remove unnecessary annotation of HostDaoImpl. While removing this annotation 
> I noticed that one of the methods were not necessary. While removing it, I 
> found some code in CloudZonesStartupProcessor that was also not used, and 
> removed it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10320) Invalid pair for response object breaking response parsing

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404488#comment-16404488
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10320:
-

marcaurele commented on issue #2481: CLOUDSTACK-10320 - Invalid pair for 
response object breaking response parsing
URL: https://github.com/apache/cloudstack/pull/2481#issuecomment-374133561
 
 
   @DaanHoogland you're right and I looked again on how to achieve this 
isolation between queries, with a `repeatable-read` but so far I didn't manage 
to have it correctly working. I read that for innodb, the default isolation in 
a transaction is the `repeatable-read` but my local debugging tests did not 
work as expected. I will dig more to find a better solution and not a hacky one.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Invalid pair for response object breaking response parsing
> --
>
> Key: CLOUDSTACK-10320
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10320
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: API
>Reporter: Marc-Aurèle Brothier
>Assignee: Marc-Aurèle Brothier
>Priority: Major
>
> Under some circumstances, the API is returning an invalid response, for 
> simplicity I will expose the JSON case. The API response on a 
> listVirtualMachines can be this string:
> {code:java}
> { "listvirtualmachinesresponse" :  ] } }{code}
> To understand how this is possible, assume you have more than one management 
> server and one is processing the destroy of a virtual machine in the account 
> X which is the only one it has. Another process is returning the result of 
> listVirtualMachines for that same account X. During the listVM command, the 
> result set is fetch with a searchAndDistinctCount due to the view 
> ([https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/query/QueryManagerImpl.java#L1024).]
>  This is done through 2 queries in the GenericDao 
> [https://github.com/apache/cloudstack/blob/master/framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java#L1323]
>  and if you encounter the _right_ conditions, the VM will be marked as 
> removed in between those 2 queries. This results in having a Pair result with 
> at least one object but a count of 0. Then following how is done the 
> serialization of the response at 
> [https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/response/ApiResponseSerializer.java#L86]
>  you will reach the case where your output is the one previously mentioned.
> To overcome this issue, there isn't a true fix but only a better pair 
> response to ensure a correct response formatting. If the result set contains 
> at least something, the count cannot be 0 but we cannot guess the correct 
> answer, but only state it has at least one element.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10320) Invalid pair for response object breaking response parsing

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404484#comment-16404484
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10320:
-

marcaurele commented on a change in pull request #2481: CLOUDSTACK-10320 - 
Invalid pair for response object breaking response parsing
URL: https://github.com/apache/cloudstack/pull/2481#discussion_r175355994
 
 

 ##
 File path: framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java
 ##
 @@ -1315,6 +1315,10 @@ protected void addJoins(StringBuilder str, 
Collection searchAndCount(final SearchCriteria sc, 
final Filter filter) {
 List objects = search(sc, filter, null, false);
 Integer count = getCount(sc);
+// Count cannot be less than the result set but can be higher due to 
pagination, see CLOUDSTACK-10320
+if (count < objects.size()) {
 
 Review comment:
   I will change how it's handled


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Invalid pair for response object breaking response parsing
> --
>
> Key: CLOUDSTACK-10320
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10320
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: API
>Reporter: Marc-Aurèle Brothier
>Assignee: Marc-Aurèle Brothier
>Priority: Major
>
> Under some circumstances, the API is returning an invalid response, for 
> simplicity I will expose the JSON case. The API response on a 
> listVirtualMachines can be this string:
> {code:java}
> { "listvirtualmachinesresponse" :  ] } }{code}
> To understand how this is possible, assume you have more than one management 
> server and one is processing the destroy of a virtual machine in the account 
> X which is the only one it has. Another process is returning the result of 
> listVirtualMachines for that same account X. During the listVM command, the 
> result set is fetch with a searchAndDistinctCount due to the view 
> ([https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/query/QueryManagerImpl.java#L1024).]
>  This is done through 2 queries in the GenericDao 
> [https://github.com/apache/cloudstack/blob/master/framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java#L1323]
>  and if you encounter the _right_ conditions, the VM will be marked as 
> removed in between those 2 queries. This results in having a Pair result with 
> at least one object but a count of 0. Then following how is done the 
> serialization of the response at 
> [https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/response/ApiResponseSerializer.java#L86]
>  you will reach the case where your output is the one previously mentioned.
> To overcome this issue, there isn't a true fix but only a better pair 
> response to ensure a correct response formatting. If the result set contains 
> at least something, the count cannot be 0 but we cannot guess the correct 
> answer, but only state it has at least one element.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10320) Invalid pair for response object breaking response parsing

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404481#comment-16404481
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10320:
-

marcaurele commented on a change in pull request #2481: CLOUDSTACK-10320 - 
Invalid pair for response object breaking response parsing
URL: https://github.com/apache/cloudstack/pull/2481#discussion_r17538
 
 

 ##
 File path: framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java
 ##
 @@ -1331,6 +1340,11 @@ protected void addJoins(StringBuilder str, 
Collection searchAndDistinctCount(final 
SearchCriteria sc, final Filter filter, final String[] distinctColumns) {
 List objects = search(sc, filter, null, false);
 Integer count = getDistinctCount(sc, distinctColumns);
+// Count cannot be 0 if there is at least a result in the list, see 
CLOUDSTACK-10320
+if (count == 0 && !objects.isEmpty()) {
+// Cannot assume if it's more than one since the count is distinct 
vs search
 
 Review comment:
   See comment before


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Invalid pair for response object breaking response parsing
> --
>
> Key: CLOUDSTACK-10320
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10320
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: API
>Reporter: Marc-Aurèle Brothier
>Assignee: Marc-Aurèle Brothier
>Priority: Major
>
> Under some circumstances, the API is returning an invalid response, for 
> simplicity I will expose the JSON case. The API response on a 
> listVirtualMachines can be this string:
> {code:java}
> { "listvirtualmachinesresponse" :  ] } }{code}
> To understand how this is possible, assume you have more than one management 
> server and one is processing the destroy of a virtual machine in the account 
> X which is the only one it has. Another process is returning the result of 
> listVirtualMachines for that same account X. During the listVM command, the 
> result set is fetch with a searchAndDistinctCount due to the view 
> ([https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/query/QueryManagerImpl.java#L1024).]
>  This is done through 2 queries in the GenericDao 
> [https://github.com/apache/cloudstack/blob/master/framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java#L1323]
>  and if you encounter the _right_ conditions, the VM will be marked as 
> removed in between those 2 queries. This results in having a Pair result with 
> at least one object but a count of 0. Then following how is done the 
> serialization of the response at 
> [https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/response/ApiResponseSerializer.java#L86]
>  you will reach the case where your output is the one previously mentioned.
> To overcome this issue, there isn't a true fix but only a better pair 
> response to ensure a correct response formatting. If the result set contains 
> at least something, the count cannot be 0 but we cannot guess the correct 
> answer, but only state it has at least one element.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10320) Invalid pair for response object breaking response parsing

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404480#comment-16404480
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10320:
-

marcaurele commented on a change in pull request #2481: CLOUDSTACK-10320 - 
Invalid pair for response object breaking response parsing
URL: https://github.com/apache/cloudstack/pull/2481#discussion_r175355499
 
 

 ##
 File path: framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java
 ##
 @@ -1323,6 +1327,11 @@ protected void addJoins(StringBuilder str, 
Collection searchAndDistinctCount(final 
SearchCriteria sc, final Filter filter) {
 List objects = search(sc, filter, null, false);
 Integer count = getDistinctCount(sc);
+// Count cannot be 0 if there is at least a result in the list, see 
CLOUDSTACK-10320
+if (count == 0 && !objects.isEmpty()) {
+// Cannot assume if it's more than one since the count is distinct 
vs search
+count = 1;
 
 Review comment:
   Moreover, the search might return a page size list, while the count should 
be global: listVirtualMachines returns a count of 234 and you have 50 entries 
in the search based on the `page` and `pageSize`.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Invalid pair for response object breaking response parsing
> --
>
> Key: CLOUDSTACK-10320
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10320
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: API
>Reporter: Marc-Aurèle Brothier
>Assignee: Marc-Aurèle Brothier
>Priority: Major
>
> Under some circumstances, the API is returning an invalid response, for 
> simplicity I will expose the JSON case. The API response on a 
> listVirtualMachines can be this string:
> {code:java}
> { "listvirtualmachinesresponse" :  ] } }{code}
> To understand how this is possible, assume you have more than one management 
> server and one is processing the destroy of a virtual machine in the account 
> X which is the only one it has. Another process is returning the result of 
> listVirtualMachines for that same account X. During the listVM command, the 
> result set is fetch with a searchAndDistinctCount due to the view 
> ([https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/query/QueryManagerImpl.java#L1024).]
>  This is done through 2 queries in the GenericDao 
> [https://github.com/apache/cloudstack/blob/master/framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java#L1323]
>  and if you encounter the _right_ conditions, the VM will be marked as 
> removed in between those 2 queries. This results in having a Pair result with 
> at least one object but a count of 0. Then following how is done the 
> serialization of the response at 
> [https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/response/ApiResponseSerializer.java#L86]
>  you will reach the case where your output is the one previously mentioned.
> To overcome this issue, there isn't a true fix but only a better pair 
> response to ensure a correct response formatting. If the result set contains 
> at least something, the count cannot be 0 but we cannot guess the correct 
> answer, but only state it has at least one element.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10320) Invalid pair for response object breaking response parsing

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404475#comment-16404475
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10320:
-

marcaurele commented on a change in pull request #2481: CLOUDSTACK-10320 - 
Invalid pair for response object breaking response parsing
URL: https://github.com/apache/cloudstack/pull/2481#discussion_r175355297
 
 

 ##
 File path: framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java
 ##
 @@ -1323,6 +1327,11 @@ protected void addJoins(StringBuilder str, 
Collection searchAndDistinctCount(final 
SearchCriteria sc, final Filter filter) {
 List objects = search(sc, filter, null, false);
 Integer count = getDistinctCount(sc);
+// Count cannot be 0 if there is at least a result in the list, see 
CLOUDSTACK-10320
+if (count == 0 && !objects.isEmpty()) {
+// Cannot assume if it's more than one since the count is distinct 
vs search
+count = 1;
 
 Review comment:
   Because the search is mostly done against views which don't return distinct 
rows due to the *not so smart* queries fetching the details rows. Look at the 
APIHelper with iterate on duplicate row entries, but only append the details.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Invalid pair for response object breaking response parsing
> --
>
> Key: CLOUDSTACK-10320
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10320
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: API
>Reporter: Marc-Aurèle Brothier
>Assignee: Marc-Aurèle Brothier
>Priority: Major
>
> Under some circumstances, the API is returning an invalid response, for 
> simplicity I will expose the JSON case. The API response on a 
> listVirtualMachines can be this string:
> {code:java}
> { "listvirtualmachinesresponse" :  ] } }{code}
> To understand how this is possible, assume you have more than one management 
> server and one is processing the destroy of a virtual machine in the account 
> X which is the only one it has. Another process is returning the result of 
> listVirtualMachines for that same account X. During the listVM command, the 
> result set is fetch with a searchAndDistinctCount due to the view 
> ([https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/query/QueryManagerImpl.java#L1024).]
>  This is done through 2 queries in the GenericDao 
> [https://github.com/apache/cloudstack/blob/master/framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java#L1323]
>  and if you encounter the _right_ conditions, the VM will be marked as 
> removed in between those 2 queries. This results in having a Pair result with 
> at least one object but a count of 0. Then following how is done the 
> serialization of the response at 
> [https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/response/ApiResponseSerializer.java#L86]
>  you will reach the case where your output is the one previously mentioned.
> To overcome this issue, there isn't a true fix but only a better pair 
> response to ensure a correct response formatting. If the result set contains 
> at least something, the count cannot be 0 but we cannot guess the correct 
> answer, but only state it has at least one element.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10320) Invalid pair for response object breaking response parsing

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404421#comment-16404421
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10320:
-

rhtyd commented on a change in pull request #2481: CLOUDSTACK-10320 - Invalid 
pair for response object breaking response parsing
URL: https://github.com/apache/cloudstack/pull/2481#discussion_r175338869
 
 

 ##
 File path: framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java
 ##
 @@ -1315,6 +1315,10 @@ protected void addJoins(StringBuilder str, 
Collection searchAndCount(final SearchCriteria sc, 
final Filter filter) {
 List objects = search(sc, filter, null, false);
 Integer count = getCount(sc);
+// Count cannot be less than the result set but can be higher due to 
pagination, see CLOUDSTACK-10320
+if (count < objects.size()) {
 
 Review comment:
   This logic may be used in changes below.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Invalid pair for response object breaking response parsing
> --
>
> Key: CLOUDSTACK-10320
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10320
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: API
>Reporter: Marc-Aurèle Brothier
>Assignee: Marc-Aurèle Brothier
>Priority: Major
>
> Under some circumstances, the API is returning an invalid response, for 
> simplicity I will expose the JSON case. The API response on a 
> listVirtualMachines can be this string:
> {code:java}
> { "listvirtualmachinesresponse" :  ] } }{code}
> To understand how this is possible, assume you have more than one management 
> server and one is processing the destroy of a virtual machine in the account 
> X which is the only one it has. Another process is returning the result of 
> listVirtualMachines for that same account X. During the listVM command, the 
> result set is fetch with a searchAndDistinctCount due to the view 
> ([https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/query/QueryManagerImpl.java#L1024).]
>  This is done through 2 queries in the GenericDao 
> [https://github.com/apache/cloudstack/blob/master/framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java#L1323]
>  and if you encounter the _right_ conditions, the VM will be marked as 
> removed in between those 2 queries. This results in having a Pair result with 
> at least one object but a count of 0. Then following how is done the 
> serialization of the response at 
> [https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/response/ApiResponseSerializer.java#L86]
>  you will reach the case where your output is the one previously mentioned.
> To overcome this issue, there isn't a true fix but only a better pair 
> response to ensure a correct response formatting. If the result set contains 
> at least something, the count cannot be 0 but we cannot guess the correct 
> answer, but only state it has at least one element.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (CLOUDSTACK-10330) Introduce a standard PULL REQUEST template

[Rohit Yadav (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rohit Yadav reassigned CLOUDSTACK-10330:


Assignee: Rohit Yadav

> Introduce a standard PULL REQUEST template
> --
>
> Key: CLOUDSTACK-10330
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10330
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Reporter: Rohit Yadav
>Assignee: Rohit Yadav
>Priority: Major
>
> This would introduce a new/standard pull request template.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10320) Invalid pair for response object breaking response parsing

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404419#comment-16404419
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10320:
-

rhtyd commented on a change in pull request #2481: CLOUDSTACK-10320 - Invalid 
pair for response object breaking response parsing
URL: https://github.com/apache/cloudstack/pull/2481#discussion_r175338811
 
 

 ##
 File path: framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java
 ##
 @@ -1323,6 +1327,11 @@ protected void addJoins(StringBuilder str, 
Collection searchAndDistinctCount(final 
SearchCriteria sc, final Filter filter) {
 List objects = search(sc, filter, null, false);
 Integer count = getDistinctCount(sc);
+// Count cannot be 0 if there is at least a result in the list, see 
CLOUDSTACK-10320
+if (count == 0 && !objects.isEmpty()) {
+// Cannot assume if it's more than one since the count is distinct 
vs search
+count = 1;
 
 Review comment:
   why not put count = objects.size() ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Invalid pair for response object breaking response parsing
> --
>
> Key: CLOUDSTACK-10320
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10320
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: API
>Reporter: Marc-Aurèle Brothier
>Assignee: Marc-Aurèle Brothier
>Priority: Major
>
> Under some circumstances, the API is returning an invalid response, for 
> simplicity I will expose the JSON case. The API response on a 
> listVirtualMachines can be this string:
> {code:java}
> { "listvirtualmachinesresponse" :  ] } }{code}
> To understand how this is possible, assume you have more than one management 
> server and one is processing the destroy of a virtual machine in the account 
> X which is the only one it has. Another process is returning the result of 
> listVirtualMachines for that same account X. During the listVM command, the 
> result set is fetch with a searchAndDistinctCount due to the view 
> ([https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/query/QueryManagerImpl.java#L1024).]
>  This is done through 2 queries in the GenericDao 
> [https://github.com/apache/cloudstack/blob/master/framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java#L1323]
>  and if you encounter the _right_ conditions, the VM will be marked as 
> removed in between those 2 queries. This results in having a Pair result with 
> at least one object but a count of 0. Then following how is done the 
> serialization of the response at 
> [https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/response/ApiResponseSerializer.java#L86]
>  you will reach the case where your output is the one previously mentioned.
> To overcome this issue, there isn't a true fix but only a better pair 
> response to ensure a correct response formatting. If the result set contains 
> at least something, the count cannot be 0 but we cannot guess the correct 
> answer, but only state it has at least one element.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10320) Invalid pair for response object breaking response parsing

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404420#comment-16404420
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10320:
-

rhtyd commented on a change in pull request #2481: CLOUDSTACK-10320 - Invalid 
pair for response object breaking response parsing
URL: https://github.com/apache/cloudstack/pull/2481#discussion_r175338839
 
 

 ##
 File path: framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java
 ##
 @@ -1331,6 +1340,11 @@ protected void addJoins(StringBuilder str, 
Collection searchAndDistinctCount(final 
SearchCriteria sc, final Filter filter, final String[] distinctColumns) {
 List objects = search(sc, filter, null, false);
 Integer count = getDistinctCount(sc, distinctColumns);
+// Count cannot be 0 if there is at least a result in the list, see 
CLOUDSTACK-10320
+if (count == 0 && !objects.isEmpty()) {
+// Cannot assume if it's more than one since the count is distinct 
vs search
 
 Review comment:
   Why not put count = objects.size() ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Invalid pair for response object breaking response parsing
> --
>
> Key: CLOUDSTACK-10320
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10320
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: API
>Reporter: Marc-Aurèle Brothier
>Assignee: Marc-Aurèle Brothier
>Priority: Major
>
> Under some circumstances, the API is returning an invalid response, for 
> simplicity I will expose the JSON case. The API response on a 
> listVirtualMachines can be this string:
> {code:java}
> { "listvirtualmachinesresponse" :  ] } }{code}
> To understand how this is possible, assume you have more than one management 
> server and one is processing the destroy of a virtual machine in the account 
> X which is the only one it has. Another process is returning the result of 
> listVirtualMachines for that same account X. During the listVM command, the 
> result set is fetch with a searchAndDistinctCount due to the view 
> ([https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/query/QueryManagerImpl.java#L1024).]
>  This is done through 2 queries in the GenericDao 
> [https://github.com/apache/cloudstack/blob/master/framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java#L1323]
>  and if you encounter the _right_ conditions, the VM will be marked as 
> removed in between those 2 queries. This results in having a Pair result with 
> at least one object but a count of 0. Then following how is done the 
> serialization of the response at 
> [https://github.com/apache/cloudstack/blob/master/server/src/main/java/com/cloud/api/response/ApiResponseSerializer.java#L86]
>  you will reach the case where your output is the one previously mentioned.
> To overcome this issue, there isn't a true fix but only a better pair 
> response to ensure a correct response formatting. If the result set contains 
> at least something, the count cannot be 0 but we cannot guess the correct 
> answer, but only state it has at least one element.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10268) Fix and enhance package script

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404417#comment-16404417
 ] 

ASF subversion and git services commented on CLOUDSTACK-10268:
--

Commit af52b1a873e51bb94a119fc4f6b375123c9299c1 in cloudstack's branch 
refs/heads/master from [~kmoossavi]
[ https://gitbox.apache.org/repos/asf?p=cloudstack.git;h=af52b1a ]

CLOUDSTACK-10268: Fix typo (#2495)

This fixes typo introduced in PR #2433 

> Fix and enhance package script
> --
>
> Key: CLOUDSTACK-10268
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10268
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Packaging
>Reporter: Khosrow Moossavi
>Priority: Minor
> Fix For: 4.12.0.0
>
>
> * new flag `-T, --use-timestamp` to use `timestamp` when POM version contains 
> SNAPSHOT
>  ** in the final artifacts (jar) name
>  ** in the final package (rpm, deb) name
>  ** in `/etc/cloudstack-release` file of SystemVMs
>  ** in the Management Server > About dialog
>  * if there's a "branding" string in the POM version (e.g. 
> `x.y.z.a-NAME[-SNAPSHOT]`),  the branding name will be used in the final 
> generated pacakge name such as following:
>   ** `cloudstack-management-x.y.z.a-NAME.NUMBER.el7.centos.x86_64`
>  ** `cloudstack-management_x.y.z.a-NAME-NUMBER~xenial_all.deb`
>  * branding string can be overriden with newly added `-b, --brand` flag
>  * handle the new format version for VR version
>  * fix long opts (they were broken)
>  * tolerate and show a warning message for unrecognized flags
>  * usage help reformat



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10330) Introduce a standard PULL REQUEST template

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404414#comment-16404414
 ] 

ASF subversion and git services commented on CLOUDSTACK-10330:
--

Commit 1ad04cbc9b165411b2e0c01cce598f1ef52b7b41 in cloudstack's branch 
refs/heads/master from [~rohithsharma]
[ https://gitbox.apache.org/repos/asf?p=cloudstack.git;h=1ad04cb ]

CLOUDSTACK-10330: Add PULL_REQUEST_TEMPLATE (#2489)

This adds a standard pull request template.

Signed-off-by: Rohit Yadav 


> Introduce a standard PULL REQUEST template
> --
>
> Key: CLOUDSTACK-10330
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10330
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Reporter: Rohit Yadav
>Priority: Major
>
> This would introduce a new/standard pull request template.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10330) Introduce a standard PULL REQUEST template

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404411#comment-16404411
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10330:
-

rhtyd closed pull request #2489: CLOUDSTACK-10330 PULL_REQUEST_TEMPLATE: Add 
pull request template
URL: https://github.com/apache/cloudstack/pull/2489
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/PULL_REQUEST_TEMPLATE.md b/PULL_REQUEST_TEMPLATE.md
new file mode 100644
index 000..8a5a30fdb54
--- /dev/null
+++ b/PULL_REQUEST_TEMPLATE.md
@@ -0,0 +1,34 @@
+## Description
+
+
+
+
+
+## Types of changes
+
+- [ ] Breaking change (fix or feature that would cause existing functionality 
to change)
+- [ ] New feature (non-breaking change which adds functionality)
+- [ ] Bug fix (non-breaking change which fixes an issue)
+- [ ] Enhancement (improves an existing feature and functionality)
+- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
+
+## Screenshots (if appropriate):
+
+## How Has This Been Tested?
+
+
+
+
+
+## Checklist:
+
+
+- [ ] I have read the 
[CONTRIBUTING](https://github.com/apache/cloudstack/blob/master/CONTRIBUTING.md)
 document.
+- [ ] My code follows the code style of this project.
+- [ ] My change requires a change to the documentation.
+- [ ] I have updated the documentation accordingly.
+- [ ] I have added tests to cover my changes.
+- [ ] All new and existing tests passed.
+
+
+@blueorangutan package


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Introduce a standard PULL REQUEST template
> --
>
> Key: CLOUDSTACK-10330
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10330
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Reporter: Rohit Yadav
>Priority: Major
>
> This would introduce a new/standard pull request template.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10330) Introduce a standard PULL REQUEST template

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404412#comment-16404412
 ] 

ASF subversion and git services commented on CLOUDSTACK-10330:
--

Commit 1ad04cbc9b165411b2e0c01cce598f1ef52b7b41 in cloudstack's branch 
refs/heads/4.11 from [~rohithsharma]
[ https://gitbox.apache.org/repos/asf?p=cloudstack.git;h=1ad04cb ]

CLOUDSTACK-10330: Add PULL_REQUEST_TEMPLATE (#2489)

This adds a standard pull request template.

Signed-off-by: Rohit Yadav 


> Introduce a standard PULL REQUEST template
> --
>
> Key: CLOUDSTACK-10330
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10330
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Reporter: Rohit Yadav
>Priority: Major
>
> This would introduce a new/standard pull request template.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10330) Introduce a standard PULL REQUEST template

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404410#comment-16404410
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10330:
-

rhtyd commented on issue #2489: CLOUDSTACK-10330 PULL_REQUEST_TEMPLATE: Add 
pull request template
URL: https://github.com/apache/cloudstack/pull/2489#issuecomment-374110764
 
 
   This is simply doc change, merging this based on feedback (incorporated) and 
2+ reviews.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Introduce a standard PULL REQUEST template
> --
>
> Key: CLOUDSTACK-10330
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10330
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Reporter: Rohit Yadav
>Priority: Major
>
> This would introduce a new/standard pull request template.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (CLOUDSTACK-10330) Introduce a standard PULL REQUEST template

[Rohit Yadav (JIRA)]

Rohit Yadav created CLOUDSTACK-10330:


 Summary: Introduce a standard PULL REQUEST template
 Key: CLOUDSTACK-10330
 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10330
 Project: CloudStack
  Issue Type: Improvement
  Security Level: Public (Anyone can view this level - this is the default.)
Reporter: Rohit Yadav


This would introduce a new/standard pull request template.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (CLOUDSTACK-10330) Introduce a standard PULL REQUEST template

[Rohit Yadav (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rohit Yadav closed CLOUDSTACK-10330.

Resolution: Fixed

> Introduce a standard PULL REQUEST template
> --
>
> Key: CLOUDSTACK-10330
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10330
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>Reporter: Rohit Yadav
>Priority: Major
>
> This would introduce a new/standard pull request template.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10310) KVM hosts reboot if there is a short transient storage error

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404409#comment-16404409
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10310:
-

rhtyd commented on a change in pull request #2472: CLOUDSTACK-10310 Fix KVM 
reboot on storage issue
URL: https://github.com/apache/cloudstack/pull/2472#discussion_r175337493
 
 

 ##
 File path: scripts/vm/hypervisor/kvm/kvmheartbeat.sh
 ##
 @@ -155,10 +155,10 @@ then
   exit 0
 elif [ "$cflag" == "1" ]
 then
-  /usr/bin/logger -t heartbeat "kvmheartbeat.sh rebooted system because it was 
unable to write the heartbeat to the storage."
+  /usr/bin/logger -t heartbeat "kvmheartbeat.sh stopped cloudstack-agent 
because it was unable to write the heartbeat to the storage."
   sync &
   sleep 5
-  echo b > /proc/sysrq-trigger
+  service cloudstack-agent stop
 
 Review comment:
   Maybe you can re-send the PR in two parts, the first half that increases the 
heartbeat retry threshold to `5` LGTM. The second part could be the 
configuration fix, global setting or setting in agent.properties to decide 
whether to reboot kvm host or stop agent.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> KVM hosts reboot if there is a short transient storage error
> 
>
> Key: CLOUDSTACK-10310
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10310
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: KVM
>Affects Versions: 4.9.0, 4.10.0.0
>Reporter: Sean Lair
>Priority: Major
>
> If the KVM heartbeat file can't be written to, the host is rebooted, and thus 
> taking down all VMs running on it.  The code does try 5x times before the 
> reboot, but the there is not a delay between the retires, so they are 5 
> simultaneous retries, which doesn't help.  Standard SAN storage HA operations 
> or quick network blip could cause this reboot to occur.
> Some discussions on the dev mailing list revealed that some people are just 
> commenting out the reboot line in their version of the CloudStack source.
> A better option (and a new PR is being issued) would be have it sleep between 
> tries so it isn't 5x almost simultaneous tries.  Plus, instead of rebooting, 
> the cloudstack-agent could just be stopped on the host instead.  This will 
> cause alerts to be issued and if the host is disconnected long-enough, 
> depending on the HA code in use, VM HA could handle the host failure.
> The built-in reboot of the host seemed drastic



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (CLOUDSTACK-10310) KVM hosts reboot if there is a short transient storage error

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404408#comment-16404408
 ] 

ASF GitHub Bot commented on CLOUDSTACK-10310:
-

rhtyd commented on a change in pull request #2472: CLOUDSTACK-10310 Fix KVM 
reboot on storage issue
URL: https://github.com/apache/cloudstack/pull/2472#discussion_r175337493
 
 

 ##
 File path: scripts/vm/hypervisor/kvm/kvmheartbeat.sh
 ##
 @@ -155,10 +155,10 @@ then
   exit 0
 elif [ "$cflag" == "1" ]
 then
-  /usr/bin/logger -t heartbeat "kvmheartbeat.sh rebooted system because it was 
unable to write the heartbeat to the storage."
+  /usr/bin/logger -t heartbeat "kvmheartbeat.sh stopped cloudstack-agent 
because it was unable to write the heartbeat to the storage."
   sync &
   sleep 5
-  echo b > /proc/sysrq-trigger
+  service cloudstack-agent stop
 
 Review comment:
   Maybe you can sent the PR in two parts, the first half that increases the 
heartbeat retry threshold to `5` LGTM. The second part could be the 
configuration fix, global setting or setting in agent.properties to decide 
whether to reboot kvm host or stop agent.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> KVM hosts reboot if there is a short transient storage error
> 
>
> Key: CLOUDSTACK-10310
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10310
> Project: CloudStack
>  Issue Type: Improvement
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: KVM
>Affects Versions: 4.9.0, 4.10.0.0
>Reporter: Sean Lair
>Priority: Major
>
> If the KVM heartbeat file can't be written to, the host is rebooted, and thus 
> taking down all VMs running on it.  The code does try 5x times before the 
> reboot, but the there is not a delay between the retires, so they are 5 
> simultaneous retries, which doesn't help.  Standard SAN storage HA operations 
> or quick network blip could cause this reboot to occur.
> Some discussions on the dev mailing list revealed that some people are just 
> commenting out the reboot line in their version of the CloudStack source.
> A better option (and a new PR is being issued) would be have it sleep between 
> tries so it isn't 5x almost simultaneous tries.  Plus, instead of rebooting, 
> the cloudstack-agent could just be stopped on the host instead.  This will 
> cause alerts to be issued and if the host is disconnected long-enough, 
> depending on the HA code in use, VM HA could handle the host failure.
> The built-in reboot of the host seemed drastic



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)