Re: [VOTE] Apache CloudStack 4.3.0 (third round)
Sure Hugo, I will enable assertions for the next run. Thanks, ~Talluri On 06/02/14 12:55 am, Hugo Trippaers trip...@gmail.com wrote: Talluri, Is it possible to enable assertions in the functional tests (-ae flag on the jvm). That way we get some feedback on the internal issues. I'm seeing some issues in master and it would be good to get that feedback from 4.3 as well. Cheers, Hugo Sent from my iPhone On 4 feb. 2014, at 14:25, Srikanteswararao Talluri srikanteswararao.tall...@citrix.com wrote: +1 No product issues observed in BVT runs. CloudStack built from commit: 8f63c2cf9c463373d774cb563e63fc56828e1418 On KVM(CentOS 6.3)/Test Run: #1185 and #1186 5 of the test failures are not valid because snapshots, scaling VM are not supported on KVM Hypervisor and more intermittent test script/env issue related to ssh. --- Link to reports: http://jenkins.buildacloud.org/job/test-smoke-matrix-4.3/1185/testReport/ http://jenkins.buildacloud.org/job/test-smoke-matrix-4.3/1186/testReport/ Total:97 Fail :5 Skip :2 name passfailskip test_loadbalance/ 3 0 0 test_guest_vlan_range/1 0 0 test_deploy_vms_with_varied_deploymentplanners/ 3 0 0 test_non_contigiousvlan/ 1 0 0 test_reset_vm_on_reboot/ 1 0 0 test_resource_detail/ 1 0 0 test_scale_vm/0 1 0 test_ssvm/ 10 0 0 test_network/ 7 0 0 test_disk_offerings/ 3 0 0 test_volumes/ 8 1 0 test_affinity_groups/ 1 0 0 test_routers/ 9 0 0 test_deploy_vm_with_userdata/ 2 0 0 test_network_acl/ 1 0 0 test_vm_life_cycle/ 10 0 0 test_privategw_acl/ 1 0 0 test_internal_lb/ 1 0 0 test_deploy_vm/ 1 0 0 test_global_settings/ 1 0 0 test_nic/ 1 0 0 test_public_ip_range/ 1 0 0 test_iso/ 5 0 1 test_service_offerings/ 4 0 0 test_multipleips_per_nic/ 1 0 0 test_regions/ 1 0 0 test_templates/ 7 0 1 test_vpc_vpn/ 2 0 0 test_vm_snapshots/0 3 0 test_portable_publicip/ 2 0 0 test_pvlan/ 1 0 0 On XenServer 6.2/Test Run: #1187 and #1188 3 failures are intermittent test script/env issues related to ssh. --- Link to reports: http://jenkins.buildacloud.org/job/test-smoke-matrix-4.3/1187/testReport/ http://jenkins.buildacloud.org/job/test-smoke-matrix-4.3/1188/testReport/ Total:97 Fail :3 Skip :2 name passfailskip test_loadbalance/ 3 0 0 test_guest_vlan_range/1 0 0 test_deploy_vms_with_varied_deploymentplanners/ 3 0 0 test_non_contigiousvlan/ 1 0 0 test_reset_vm_on_reboot/ 1 0 0 test_resource_detail/ 1 0 0 test_scale_vm/1 0 0 test_ssvm/ 10 0 0 test_network/ 7 0 0 test_disk_offerings/ 3 0 0 test_volumes/ 9 0 0 test_affinity_groups/ 1 0 0 test_routers/ 9 0 0 test_deploy_vm_with_userdata/ 2 0 0 test_network_acl/ 1 0 0 test_vm_life_cycle/
Re: User UI, ACL configuration
On Feb 6, 2014, at 11:53 PM, d...@soleks.com wrote: it's 4.2.1 bug ? Dan, definitely. It may also apply to 4.3, so please tag both thanks On Feb 6, 2014, at 3:31 PM, d...@soleks.com wrote: Hi @dev, There is some minor issue with UI, i think it's controlled by some settings (need heads up where is it). So in the firewall configuration for IP Addresses long CIDRs are displayed as 192.70.132... That is not conviniet becuase totaly it's totally hiding settings. The field need to be wider - +3-4 characters. If somebody can gimme direction where to fix it i'd highly appriciate it. Dan. Dan is this on 4.3 ? In any case please file a bug. thanks This message was sent using IMP, the Internet Messaging Program. This message was sent using IMP, the Internet Messaging Program.
RE: commit 6523c068695d0431070060667c222eb40d54b14d breaks network removal
-Original Message- From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] Sent: Thursday, February 06, 2014 9:56 PM To: Alena Prokharchyk Cc: dev@cloudstack.apache.org; eiz...@infoblox.com Subject: Re: commit 6523c068695d0431070060667c222eb40d54b14d breaks network removal second thought, Soheils mail bounces and the commit does not refer a ticket from jira. I am going to revert. I should have been more vigilant. sorry. [Animesh] Daan was this change introduced because of being flagged during findbug run? On Fri, Feb 7, 2014 at 6:49 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: will do Alena, thanks for the headsup On Thu, Feb 6, 2014 at 10:42 PM, Alena Prokharchyk alena.prokharc...@citrix.com wrote: Soheil/Daan, The commit in the subject breaks network System vms destroy (VR, SSVM, CPVM), resulting in the network removal failures. Following line replacement causes the failure: - if (vm.getType() == Type.User isDhcpAccrossMultipleSubnetsSupported(network) isLastNicInSubnet(nic) network.getTrafficType() == TrafficType.Guest With +DhcpServiceProvider dhcpServiceProvider = getDhcpServiceProvider(network); When you try to call getDhcpServiceProvider(network), it throws an exception because DHCP service is not enabled in Public/Control networks of system vms nics. So system vm always fails to expunge. Could you please fix it by checking if DHCP service is enabled on the network, before getting the DHCP service provider? Thanks, Alena. -- Daan -- Daan
Re: Code quality, QA, etc
Hey David, I would make a distinction between code issues and functional issues. Occasionally somebody just plainly breaks the build, i’m guilty of that myself actually, and thats just plain stupid. Luckily we have Jenkins to catch these errors quickly. I’m in a continuous struggle with Jenkins to get the build time to less than 5 minutes. I feel that is an acceptable time to get feedback on a commit, any longer and you have moved on to the next thing or gone home. Also this kind of testing isn’t really hard, run the build and unit tests. By introducing something like gerrit we can actually make this happen before committing it to the repo. Push a patch to gerrit, gerrit tells jenkins to test the patch, if +1 from jerkins commit, for non committers the step would be to invite somebody for review as well. Second nice thing about jenkins is the post-review test, if a contributor submits a patch its build by jenkins, if a reviewes approves the patch, jerkins will again run a build to ensure that the patch will still apply and doesn’t break the build. Very handy if there is some time between patch submission and patch review. Functional issues are much harder to track. For example yesterday i found several issues in the contrail plugin that would not cause any pain in a contrail environment, but any other environments creating a network would fail. These examples are too common and difficult to catch with unit tests. It can be done, but requires some serious effort on the developers side and we in general don’t seem to be very active at writing unit tests. These kind of issues can only be found by actually running CloudStack and executing a series of functional tests. Ideally that is what we have the BVT suite for, but i think our current BVT setup is not documented enough to give accurate feedback to a developer about which patch broke a certain piece of functionality. In jenkins the path from code to BVT is not kept yet, so it is almost impossible to see which commits were new in a particular run of the bvt suite. Personally i’m trying to get into the habit of running a series of tests on devcloud before committing something. Doesn't prove a lot, but does guarantee that the bare basic developer functionality is working before committing something. After a commit at least i’m sure that anybody will be able to spinup devcloud and deploy an instance. I’m trying to get this automated as well so we can use this as feedback on a patch. Beers for anyone who writes an easy to use script that configures devcloud with a zone and tests if a user vm can be instantiated on an isolated sourcenat network. If we could include such a script in the tree it might help people with testing their patch before committing. I think we are seeing more and more reverts in the tree. Not necessarily a good thing, but at least people know that there is that option if a commit really breaks a build. Also please help each other out, everybody can make a mistake and commit it. If its a trivial mistake it might not be much effort to track it down and fix it, which is way better than a revert or a mail that something is broken. In short, we need to make testing more efficient and transparent to allow people to easily incorporate it in their personal workflow. Cheers, Hugo On 7 feb. 2014, at 04:50, David Nalley da...@gnsa.us wrote: Hi folks, We continue to break things large and small in the codebase, and after a number of different conversations; I thought I'd bring that discussion here. First - coding quality is only one factor that the PMC considers when making someone a committer. Second - CloudStack is a huge codebase; has a ton of inter-related pieces, and unintended consequences are easy. We also have an pretty heady commit velocity - 20+ commits today alone. Some communities have Review-then-commit - which would slow us down, and presumably help us increase quality. However, I am not personally convinced that it will do so measurably because even the most experienced CloudStack developers occasionally break a build or worse. We could have an automated pipeline that verifies a number of different tests pass - before a patch/commit makes it into a mainline branch. That is difficult with our current tooling; but perhaps something worth considering. At FOSDEM, Hugo and I were discussing his experiences with Gerrit and OpenDaylight, and he thinks thats a viable option. I think it would certainly be a step in the right direction. Separately, Jake Farrell and I were discussing our git-related proposal for ApacheCon, and broached the subject of Gerrit. Jake is the current person bearing most of the load for git at the ASF, and he's also run Gerrit in other contexts. He points out a number of difficulties. (And I'd love for him to weigh in on this conversation, hence the CC) He wants to expand RB significantly, including pre-commit testing. So -
Re: commit 6523c068695d0431070060667c222eb40d54b14d breaks network removal
On Fri, Feb 7, 2014 at 11:34 AM, Daan Hoogland daan.hoogl...@gmail.comwrote:
Alena,
The revert didn't apply. Would the folowing do the trick?
if (vm.getType() == Type.User
network.getTrafficType() == TrafficType.Guest
network.getGuestType() == GuestType.Shared) {
// remove the dhcpservice ip if this is the last nic in subnet.
DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
if (dhcpServiceProvider != null
isDhcpAccrossMultipleSubnetsSupported(dhcpServiceProvider)
isLastNicInSubnet(nic)) {
removeDhcpServiceInSubnet(nic);
}
}
Daan, it would still break if network does not have DHCP service enabled,
best would be to check
'_networkModel.areServicesSupportedInNetwork(network.getId(),
Service.Dhcp)' then get the provider.
On Fri, Feb 7, 2014 at 6:55 AM, Daan Hoogland daan.hoogl...@gmail.com
wrote:
second thought,
Soheils mail bounces and the commit does not refer a ticket from jira.
I am going to revert. I should have been more vigilant. sorry.
On Fri, Feb 7, 2014 at 6:49 AM, Daan Hoogland daan.hoogl...@gmail.com
wrote:
will do Alena,
thanks for the headsup
On Thu, Feb 6, 2014 at 10:42 PM, Alena Prokharchyk
alena.prokharc...@citrix.com wrote:
Soheil/Daan,
The commit in the subject breaks network System vms destroy (VR, SSVM,
CPVM), resulting in the network removal failures. Following line
replacement
causes the failure:
- if (vm.getType() == Type.User
isDhcpAccrossMultipleSubnetsSupported(network)
isLastNicInSubnet(nic)
network.getTrafficType() == TrafficType.Guest
With
+DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
When you try to call getDhcpServiceProvider(network), it throws an
exception
because DHCP service is not enabled in Public/Control networks of
system vms
nics. So system vm always fails to expunge.
Could you please fix it by checking if DHCP service is enabled on the
network, before getting the DHCP service provider?
Thanks,
Alena.
--
Daan
--
Daan
--
Daan
Re: Removing deploy\load options from marvinplugin
On Fri, Feb 07, 2014 at 07:25:03AM +, Santhosh Edukulla wrote: 1. code restructuring ,definitely yes, it makes little neat and plugin does not worry much about deploy altogether. Take an example of load option, it is little redundant i believe, if user passes the deploy flag, deploy should work and continue, if not passed should be treated as work with loading provided configuration and continue with no deploy. May be make the plugin smarter and include less options then? Even, for redeploying, user can still use deployDC, we don't exit cleanly in a way if deployDC has an issue. Again - feels like an area of improvement for deploydatacenter reason behind this is providing some fine tuner logging for test modules not worrying about the logs when deployDC runs as part of marvinplugin. I have seen people currently run individual test suites post deployDC separately. deploydatacenter failures could use logging. what is fine-tuned logging? our test modules have their own logs correct? may be the logger configuration should be outside the deployer, is this what you mean? Is there a case explicitly for redploying with same configuration and i believe if so it breaks, if its a new cofiguration then its a new deploy altogether. To make plugin init, start cleaner this makes to remove.Tying nosetests plugin to few things other than tests is also little confusing. A bit of history - the reason the load option is present is because you can't start running the tests immediately after deploying your cloud. This actually sucks. The reason for the two step - deploy and run tests is because certain configurations in the global settings need a restart of the cloudstack service. The original test runner was meant to run tests immediately after deploy. Hence, when load is not specified, it starts running tests immediately. 2. Exporting a datacenter option would be an idea i believe best fits for cloudstack, this i have raised in a mail thread earlier, where user can export or import configuration for a datacenter. So, that once exported can tweak few parameters and reimport. This will help create second datacenter with similar configuration easy and store his existing configuration as well, it will help other areas of automation as well. Right - there's a JIRA at CLOUDSTACK-4590 - I see the deployer being part of marvin for this reason. Different configurations, same tests. Same configuration, different tests. 3. Not sure, why we need to use marvinplugin for simulator deployDC? we can still use deployDC and run tests against the similator. Currently, also i have seen we use unittest load and run as part of mvn profile for simulator. Any places where marvinplugin is used for simulator? For devcloud, do we run nosetests using marvinplugin or run deployDC separately, if its a case we can remove there as well. marvinplugin is only a console entry point for nose really. it is basically calling in deploydatacenter. just a nice option. however, i don't see how it hinders anything. hence my surprise at removing it. -- Prasanna.,
Re: Review Request 17736: CLOUDSTACK-5999: Virtual Router does not start if Guest VM is rebooted from CloudStack
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17736/#review33921 --- server/src/com/cloud/vm/UserVmManagerImpl.java https://reviews.apache.org/r/17736/#comment63715 Thanks John for the comments. Created single list and now using new utility to grab the domain router for the network. - Saksham Srivastava On Feb. 5, 2014, 8:46 a.m., Saksham Srivastava wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17736/ --- (Updated Feb. 5, 2014, 8:46 a.m.) Review request for cloudstack and Murali Reddy. Bugs: CLOUDSTACK-5999 https://issues.apache.org/jira/browse/CLOUDSTACK-5999 Repository: cloudstack-git Description --- When a guest is rebooted from CloudStack, if the virtual router managing the guest network of that guest is down, CloudStack will not start the virtual router. However the router is started in case the guest vm is stopped and then started. To mantain similarity between the 2 process it is necessary to start the VR in case it is not running. The fix will address the same. Diffs - server/src/com/cloud/vm/UserVmManagerImpl.java 81be19f Diff: https://reviews.apache.org/r/17736/diff/ Testing --- Testing: 1) vm in a single guest network : vm Reboot : If the VR is stopped: VR is first started and then the VM reboots. vm Reboot : If the VR is running, VM reboots as it used to. vm Stop/Start continue to work the same. 2) vm having nics in multi networks : vm Reboot : If VR in any/all network is stopped: VRs are first started and then the VM reboots. vm Reboot : If the VRs are running, VM reboots as it used to. vm Stop/Start continue to work the same. 3) vpc : Tested the above scenarios for vpc also, works fine. Patch applies cleanly. Thanks, Saksham Srivastava
Re: Review Request 17736: CLOUDSTACK-5999: Virtual Router does not start if Guest VM is rebooted from CloudStack
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17736/ --- (Updated Feb. 7, 2014, 9:26 a.m.) Review request for cloudstack, John Burwell and Murali Reddy. Changes --- Updated the patch. Bugs: CLOUDSTACK-5999 https://issues.apache.org/jira/browse/CLOUDSTACK-5999 Repository: cloudstack-git Description --- When a guest is rebooted from CloudStack, if the virtual router managing the guest network of that guest is down, CloudStack will not start the virtual router. However the router is started in case the guest vm is stopped and then started. To mantain similarity between the 2 process it is necessary to start the VR in case it is not running. The fix will address the same. Diffs (updated) - server/src/com/cloud/vm/UserVmManagerImpl.java 81be19f Diff: https://reviews.apache.org/r/17736/diff/ Testing --- Testing: 1) vm in a single guest network : vm Reboot : If the VR is stopped: VR is first started and then the VM reboots. vm Reboot : If the VR is running, VM reboots as it used to. vm Stop/Start continue to work the same. 2) vm having nics in multi networks : vm Reboot : If VR in any/all network is stopped: VRs are first started and then the VM reboots. vm Reboot : If the VRs are running, VM reboots as it used to. vm Stop/Start continue to work the same. 3) vpc : Tested the above scenarios for vpc also, works fine. Patch applies cleanly. Thanks, Saksham Srivastava
Re: Review Request 15932: Add support for Primary Storage on Gluster using the libvirt backend
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15932/#review33922 --- My sincere apologies Niels! I completely missed the second version of your patch! I just took a look at it again and it seems pretty straight forward. Most of the RBD code did a lot of work for you, so it's fairly easy to have GlusterFS in CS. I tried applying the patch to the master branch and that failed. It seems that you wrote the patch against the 4.2 branch, correct? Could you try to rebase it again master? If it then applies we might be able to get GlusterFS into 4.4! :) - Wido den Hollander On Jan. 14, 2014, 3:54 p.m., Niels de Vos wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15932/ --- (Updated Jan. 14, 2014, 3:54 p.m.) Review request for cloudstack. Repository: cloudstack-git Description --- The support for Gluster as Primary Storage is mostly based on the implementation for NFS. Like NFS, libvirt can address a Gluster environment through the 'netfs' pool-type. Diffs - api/src/com/cloud/storage/Storage.java 07b6667 plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java 182cb22 plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtStoragePoolDef.java e181cea plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtStoragePoolXMLParser.java a707a0b plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtVMDef.java 6aaabc5 plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java aaefc16 plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java 0760e51 plugins/storage/volume/default/src/org/apache/cloudstack/storage/datastore/lifecycle/CloudStackPrimaryDataStoreLifeCycleImpl.java 7555c1e Diff: https://reviews.apache.org/r/15932/diff/ Testing --- See http://blog.nixpanic.net/2013/12/using-gluster-as-primary-storage-in.html Thanks, Niels de Vos
RE: Removing deploy\load options from marvinplugin
CIL From: Prasanna Santhanam [t...@apache.org] Sent: Friday, February 07, 2014 4:18 AM To: dev@cloudstack.apache.org Subject: Re: Removing deploy\load options from marvinplugin On Fri, Feb 07, 2014 at 07:25:03AM +, Santhosh Edukulla wrote: 1. code restructuring ,definitely yes, it makes little neat and plugin does not worry much about deploy altogether. Take an example of load option, it is little redundant i believe, if user passes the deploy flag, deploy should work and continue, if not passed should be treated as work with loading provided configuration and continue with no deploy. May be make the plugin smarter and include less options then? [Santhosh] : I will remove load option. Even, for redeploying, user can still use deployDC, we don't exit cleanly in a way if deployDC has an issue. Again - feels like an area of improvement for deploydatacenter reason behind this is providing some fine tuner logging for test modules not worrying about the logs when deployDC runs as part of marvinplugin. I have seen people currently run individual test suites post deployDC separately. deploydatacenter failures could use logging. what is fine-tuned logging? our test modules have their own logs correct? may be the logger configuration should be outside the deployer, is this what you mean? Is there a case explicitly for redploying with same configuration and i believe if so it breaks, if its a new cofiguration then its a new deploy altogether. To make plugin init, start cleaner this makes to remove.Tying nosetests plugin to few things other than tests is also little confusing. A bit of history - the reason the load option is present is because you can't start running the tests immediately after deploying your cloud. This actually sucks. The reason for the two step - deploy and run tests is because certain configurations in the global settings need a restart of the cloudstack service. The original test runner was meant to run tests immediately after deploy. Hence, when load is not specified, it starts running tests immediately. [Santhosh] : Its still little unclear here, using load explicitly as we dont have a sequence maintained for restarting cs and we are loading from config. 2. Exporting a datacenter option would be an idea i believe best fits for cloudstack, this i have raised in a mail thread earlier, where user can export or import configuration for a datacenter. So, that once exported can tweak few parameters and reimport. This will help create second datacenter with similar configuration easy and store his existing configuration as well, it will help other areas of automation as well. Right - there's a JIRA at CLOUDSTACK-4590 - I see the deployer being part of marvin for this reason. Different configurations, same tests. Same configuration, different tests. [Santhosh] : Frankly, i dont see a case to export the configuration, we are creating deployDC using a config signifies we have a config, why export the same using marvin again? 3. Not sure, why we need to use marvinplugin for simulator deployDC? we can still use deployDC and run tests against the similator. Currently, also i have seen we use unittest load and run as part of mvn profile for simulator. Any places where marvinplugin is used for simulator? For devcloud, do we run nosetests using marvinplugin or run deployDC separately, if its a case we can remove there as well. marvinplugin is only a console entry point for nose really. it is basically calling in deploydatacenter. just a nice option. however, i don't see how it hinders anything. hence my surprise at removing it. [Santhosh]: Again it seems we are not using marvinplugin here, so in a way it wont these areas, to double confirm? -- Prasanna.,
Review Request 17840: Patch [master] CLOUDSTACK-5999: Virtual Router does not start if Guest VM is rebooted from CloudStack
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17840/ --- Review request for cloudstack, John Burwell and Murali Reddy. Bugs: CLOUDSTACK-5999 https://issues.apache.org/jira/browse/CLOUDSTACK-5999 Repository: cloudstack-git Description --- Patch for matser for bug CLOUDSTACK-5999. The parent review request is at https://reviews.apache.org/r/17736/ Diffs - server/src/com/cloud/vm/UserVmManagerImpl.java f6f20cc Diff: https://reviews.apache.org/r/17840/diff/ Testing --- Patch applies cleanly. Build passes. Thanks, Saksham Srivastava
Re: Removing deploy\load options from marvinplugin
On Fri, Feb 07, 2014 at 09:36:49AM +, Santhosh Edukulla wrote: From: Prasanna Santhanam [t...@apache.org] Sent: Friday, February 07, 2014 4:18 AM To: dev@cloudstack.apache.org Subject: Re: Removing deploy\load options from marvinplugin On Fri, Feb 07, 2014 at 07:25:03AM +, Santhosh Edukulla wrote: 1. code restructuring ,definitely yes, it makes little neat and plugin does not worry much about deploy altogether. Take an example of load option, it is little redundant i believe, if user passes the deploy flag, deploy should work and continue, if not passed should be treated as work with loading provided configuration and continue with no deploy. May be make the plugin smarter and include less options then? [Santhosh] : I will remove load option. What would the new behaviour be? nosetests only runs tests? and user has to do deploydatacenter before? reason behind this is providing some fine tuner logging for test modules not worrying about the logs when deployDC runs as part of marvinplugin. I have seen people currently run individual test suites post deployDC separately. deploydatacenter failures could use logging. what is fine-tuned logging? our test modules have their own logs correct? may be the logger configuration should be outside the deployer, is this what you mean? Can you answer this issue about logging you had raised? Didn't quite understand what you said. Is there a case explicitly for redploying with same configuration and i believe if so it breaks, if its a new cofiguration then its a new deploy altogether. To make plugin init, start cleaner this makes to remove.Tying nosetests plugin to few things other than tests is also little confusing. A bit of history - the reason the load option is present is because you can't start running the tests immediately after deploying your cloud. This actually sucks. The reason for the two step - deploy and run tests is because certain configurations in the global settings need a restart of the cloudstack service. The original test runner was meant to run tests immediately after deploy. Hence, when load is not specified, it starts running tests immediately. [Santhosh] : Its still little unclear here, using load explicitly as we dont have a sequence maintained for restarting cs and we are loading from config. I dont think you follow - users are doing the two steps because global settings are required before starting tests - and that requires a restart. So by retaining or removing, the users are not going to benefit from this. They'll use 4 different steps after this. How's that a simplification? 1. deploy cloudstack 2. deploydatacenter (done using nose earlier) 3. restart cloudstack 4. run tests (also done by nose earlier) Would including the restart within the plugin not make it a single step? What do you think? 2. Exporting a datacenter option would be an idea i believe best fits for cloudstack, this i have raised in a mail thread earlier, where user can export or import configuration for a datacenter. So, that once exported can tweak few parameters and reimport. This will help create second datacenter with similar configuration easy and store his existing configuration as well, it will help other areas of automation as well. Right - there's a JIRA at CLOUDSTACK-4590 - I see the deployer being part of marvin for this reason. Different configurations, same tests. Same configuration, different tests. [Santhosh] : Frankly, i dont see a case to export the configuration, we are creating deployDC using a config signifies we have a config, why export the same using marvin again? Why would an IaaS need to leak information about itself? That is totally up to cloudstack. What we could do is query, that's easier to include than including an API within CS and later templatize it for reuse. Isn't repeating testbed configuration a good benefit? 3. Not sure, why we need to use marvinplugin for simulator deployDC? we can still use deployDC and run tests against the similator. Currently, also i have seen we use unittest load and run as part of mvn profile for simulator. Any places where marvinplugin is used for simulator? For devcloud, do we run nosetests using marvinplugin or run deployDC separately, if its a case we can remove there as well. marvinplugin is only a console entry point for nose really. it is basically calling in deploydatacenter. just a nice option. however, i don't see how it hinders anything. hence my surprise at removing it. [Santhosh]: Again it seems we are not using marvinplugin here, so in a way it wont these areas, to double confirm? true - it won't affect these specific areas. but the deployer is nice to have in the same tool that runs the tests. splitting it causes an additional headache of maintaining an additional module elsewhere in the repo. Powered by BigRock.com
Re: Review Request 15932: Add support for Primary Storage on Gluster using the libvirt backend
On Feb. 7, 2014, 10:33 a.m., Wido den Hollander wrote: My sincere apologies Niels! I completely missed the second version of your patch! I just took a look at it again and it seems pretty straight forward. Most of the RBD code did a lot of work for you, so it's fairly easy to have GlusterFS in CS. I tried applying the patch to the master branch and that failed. It seems that you wrote the patch against the 4.2 branch, correct? Could you try to rebase it again master? If it then applies we might be able to get GlusterFS into 4.4! :) Sure, no problem! I've done a rebase against 4.3 already, moving to master or 4.4 should not take too much time. - Niels --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15932/#review33922 --- On Jan. 14, 2014, 4:54 p.m., Niels de Vos wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15932/ --- (Updated Jan. 14, 2014, 4:54 p.m.) Review request for cloudstack. Repository: cloudstack-git Description --- The support for Gluster as Primary Storage is mostly based on the implementation for NFS. Like NFS, libvirt can address a Gluster environment through the 'netfs' pool-type. Diffs - api/src/com/cloud/storage/Storage.java 07b6667 plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java 182cb22 plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtStoragePoolDef.java e181cea plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtStoragePoolXMLParser.java a707a0b plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtVMDef.java 6aaabc5 plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java aaefc16 plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java 0760e51 plugins/storage/volume/default/src/org/apache/cloudstack/storage/datastore/lifecycle/CloudStackPrimaryDataStoreLifeCycleImpl.java 7555c1e Diff: https://reviews.apache.org/r/15932/diff/ Testing --- See http://blog.nixpanic.net/2013/12/using-gluster-as-primary-storage-in.html Thanks, Niels de Vos
Re: Review Request 16385: Fix for CloudStack JIRA 4406
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/16385/ --- (Updated Feb. 7, 2014, 10:30 a.m.) Review request for cloudstack and daan Hoogland. Changes --- Daan, I just updated the repo and uploaded the diff against the latest master. No additional code changes have been made to the original patch other than the comment describing the abstract function. Thanks, Mandar Bugs: CLOUDSTACK-4406 https://issues.apache.org/jira/browse/CLOUDSTACK-4406 Repository: cloudstack-git Description --- JIRA 4406 expects removal of cleanString() call for performance improvements. This is called when building audit trail for command responses and used for removing sensitive data (passwords, secret keys) from the log buffer. All the API responses do not carry such sensitive information so pattern matching done by cleanString against all API response strings can be costly. I propose following for a solution: * Modify BaseCmd class to add flags that will store cmd/response sensitivity * At init these flags will be set to false indicating no cmd req/resp carries sensitive data * any child api cmd class that will carry sensitive data in the req/resp should set the respective flags * before calling any logging function the flag should be checked and cleanString should be called only for cmds with flags set Pro: This approach will scale well as new cmds get added and no additional changes should be required. Con: Big change upfront as it will touch all API cmd classes that carry sensitive information along with BaseCmd class. NOTE: changes should be simple and straightforward though spread across multiple classes. Diffs (updated) - api/src/com/cloud/api/commands/ListRecurringSnapshotScheduleCmd.java d34c09c api/src/org/apache/cloudstack/api/BaseCmd.java 0cfb950 api/src/org/apache/cloudstack/api/BaseListTemplateOrIsoPermissionsCmd.java 48c1e02 api/src/org/apache/cloudstack/api/command/admin/account/CreateAccountCmd.java c5a2d1a api/src/org/apache/cloudstack/api/command/admin/account/DeleteAccountCmd.java 7c1b206 api/src/org/apache/cloudstack/api/command/admin/account/DisableAccountCmd.java 6fdbefe api/src/org/apache/cloudstack/api/command/admin/account/EnableAccountCmd.java 59d6acd api/src/org/apache/cloudstack/api/command/admin/account/LockAccountCmd.java 93ec1be api/src/org/apache/cloudstack/api/command/admin/account/UpdateAccountCmd.java a8cf63f api/src/org/apache/cloudstack/api/command/admin/alert/GenerateAlertCmd.java 620c5ed api/src/org/apache/cloudstack/api/command/admin/autoscale/CreateCounterCmd.java 6c4b81b api/src/org/apache/cloudstack/api/command/admin/autoscale/DeleteCounterCmd.java 50477f5 api/src/org/apache/cloudstack/api/command/admin/cluster/AddClusterCmd.java d0e7380 api/src/org/apache/cloudstack/api/command/admin/cluster/DeleteClusterCmd.java e1bc585 api/src/org/apache/cloudstack/api/command/admin/cluster/ListClustersCmd.java 8640f37 api/src/org/apache/cloudstack/api/command/admin/cluster/UpdateClusterCmd.java b13f81a api/src/org/apache/cloudstack/api/command/admin/config/ListCfgsByCmd.java 517807d api/src/org/apache/cloudstack/api/command/admin/config/ListDeploymentPlannersCmd.java 1d9d2d9 api/src/org/apache/cloudstack/api/command/admin/config/ListHypervisorCapabilitiesCmd.java 16adf66 api/src/org/apache/cloudstack/api/command/admin/config/UpdateCfgCmd.java 9bc9b3c api/src/org/apache/cloudstack/api/command/admin/config/UpdateHypervisorCapabilitiesCmd.java 5cb5f9c api/src/org/apache/cloudstack/api/command/admin/domain/CreateDomainCmd.java 4737555 api/src/org/apache/cloudstack/api/command/admin/domain/DeleteDomainCmd.java b1075c1 api/src/org/apache/cloudstack/api/command/admin/domain/ListDomainChildrenCmd.java e1ba178 api/src/org/apache/cloudstack/api/command/admin/domain/ListDomainsCmd.java 5a3786c api/src/org/apache/cloudstack/api/command/admin/domain/UpdateDomainCmd.java 8acfcd5 api/src/org/apache/cloudstack/api/command/admin/host/AddHostCmd.java 363bcd6 api/src/org/apache/cloudstack/api/command/admin/host/AddSecondaryStorageCmd.java 61f6f49 api/src/org/apache/cloudstack/api/command/admin/host/CancelMaintenanceCmd.java 46289ee api/src/org/apache/cloudstack/api/command/admin/host/DeleteHostCmd.java 5a4478e api/src/org/apache/cloudstack/api/command/admin/host/FindHostsForMigrationCmd.java 0faf72c api/src/org/apache/cloudstack/api/command/admin/host/ListHostsCmd.java eda821b api/src/org/apache/cloudstack/api/command/admin/host/PrepareForMaintenanceCmd.java 23cfacf api/src/org/apache/cloudstack/api/command/admin/host/ReconnectHostCmd.java 1ce888b api/src/org/apache/cloudstack/api/command/admin/host/ReleaseHostReservationCmd.java b60feca
[DISCUSS]null pointer returned from daoimpl
LS,
in PrivateIpDaoImpl a null pointer is returned when no db object can be found:
@Override
public PrivateIpVO allocateIpAddress(long dcId, long networkId,
String requestedIp) {
...
PrivateIpVO vo = lockOneRandomRow(sc, true);
if (vo == null) {
txn.rollback();
return null;
}
...
return vo;
}
I would expect it to throw a ClodException of some sort and would like
to change it to that but recognize that the null pointer could be of
significance in cases. Is there a policy on how dao's should return
failures?
In my opinion a null should never be returned by a dao, at most a vo
containing a null but this seldom makes sense.
--
Daan
Re: Review Request 16385: Fix for CloudStack JIRA 4406
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/16385/ --- (Updated Feb. 7, 2014, 10:30 a.m.) Review request for cloudstack and daan Hoogland. Changes --- Daan, I just updated the repo and uploaded the diff against the latest master. No additional code changes have been made to the original patch other than the comment describing the abstract function. Thanks, Mandar Bugs: CLOUDSTACK-4406 https://issues.apache.org/jira/browse/CLOUDSTACK-4406 Repository: cloudstack-git Description --- JIRA 4406 expects removal of cleanString() call for performance improvements. This is called when building audit trail for command responses and used for removing sensitive data (passwords, secret keys) from the log buffer. All the API responses do not carry such sensitive information so pattern matching done by cleanString against all API response strings can be costly. I propose following for a solution: * Modify BaseCmd class to add flags that will store cmd/response sensitivity * At init these flags will be set to false indicating no cmd req/resp carries sensitive data * any child api cmd class that will carry sensitive data in the req/resp should set the respective flags * before calling any logging function the flag should be checked and cleanString should be called only for cmds with flags set Pro: This approach will scale well as new cmds get added and no additional changes should be required. Con: Big change upfront as it will touch all API cmd classes that carry sensitive information along with BaseCmd class. NOTE: changes should be simple and straightforward though spread across multiple classes. Diffs - api/src/com/cloud/api/commands/ListRecurringSnapshotScheduleCmd.java d34c09c api/src/org/apache/cloudstack/api/BaseCmd.java 0cfb950 api/src/org/apache/cloudstack/api/BaseListTemplateOrIsoPermissionsCmd.java 48c1e02 api/src/org/apache/cloudstack/api/command/admin/account/CreateAccountCmd.java c5a2d1a api/src/org/apache/cloudstack/api/command/admin/account/DeleteAccountCmd.java 7c1b206 api/src/org/apache/cloudstack/api/command/admin/account/DisableAccountCmd.java 6fdbefe api/src/org/apache/cloudstack/api/command/admin/account/EnableAccountCmd.java 59d6acd api/src/org/apache/cloudstack/api/command/admin/account/LockAccountCmd.java 93ec1be api/src/org/apache/cloudstack/api/command/admin/account/UpdateAccountCmd.java a8cf63f api/src/org/apache/cloudstack/api/command/admin/alert/GenerateAlertCmd.java 620c5ed api/src/org/apache/cloudstack/api/command/admin/autoscale/CreateCounterCmd.java 6c4b81b api/src/org/apache/cloudstack/api/command/admin/autoscale/DeleteCounterCmd.java 50477f5 api/src/org/apache/cloudstack/api/command/admin/cluster/AddClusterCmd.java d0e7380 api/src/org/apache/cloudstack/api/command/admin/cluster/DeleteClusterCmd.java e1bc585 api/src/org/apache/cloudstack/api/command/admin/cluster/ListClustersCmd.java 8640f37 api/src/org/apache/cloudstack/api/command/admin/cluster/UpdateClusterCmd.java b13f81a api/src/org/apache/cloudstack/api/command/admin/config/ListCfgsByCmd.java 517807d api/src/org/apache/cloudstack/api/command/admin/config/ListDeploymentPlannersCmd.java 1d9d2d9 api/src/org/apache/cloudstack/api/command/admin/config/ListHypervisorCapabilitiesCmd.java 16adf66 api/src/org/apache/cloudstack/api/command/admin/config/UpdateCfgCmd.java 9bc9b3c api/src/org/apache/cloudstack/api/command/admin/config/UpdateHypervisorCapabilitiesCmd.java 5cb5f9c api/src/org/apache/cloudstack/api/command/admin/domain/CreateDomainCmd.java 4737555 api/src/org/apache/cloudstack/api/command/admin/domain/DeleteDomainCmd.java b1075c1 api/src/org/apache/cloudstack/api/command/admin/domain/ListDomainChildrenCmd.java e1ba178 api/src/org/apache/cloudstack/api/command/admin/domain/ListDomainsCmd.java 5a3786c api/src/org/apache/cloudstack/api/command/admin/domain/UpdateDomainCmd.java 8acfcd5 api/src/org/apache/cloudstack/api/command/admin/host/AddHostCmd.java 363bcd6 api/src/org/apache/cloudstack/api/command/admin/host/AddSecondaryStorageCmd.java 61f6f49 api/src/org/apache/cloudstack/api/command/admin/host/CancelMaintenanceCmd.java 46289ee api/src/org/apache/cloudstack/api/command/admin/host/DeleteHostCmd.java 5a4478e api/src/org/apache/cloudstack/api/command/admin/host/FindHostsForMigrationCmd.java 0faf72c api/src/org/apache/cloudstack/api/command/admin/host/ListHostsCmd.java eda821b api/src/org/apache/cloudstack/api/command/admin/host/PrepareForMaintenanceCmd.java 23cfacf api/src/org/apache/cloudstack/api/command/admin/host/ReconnectHostCmd.java 1ce888b api/src/org/apache/cloudstack/api/command/admin/host/ReleaseHostReservationCmd.java b60feca
Re: commit 6523c068695d0431070060667c222eb40d54b14d breaks network removal
thanks Murali, will do
On Fri, Feb 7, 2014 at 9:58 AM, murali reddy muralimmre...@gmail.com wrote:
On Fri, Feb 7, 2014 at 11:34 AM, Daan Hoogland daan.hoogl...@gmail.comwrote:
Alena,
The revert didn't apply. Would the folowing do the trick?
if (vm.getType() == Type.User
network.getTrafficType() == TrafficType.Guest
network.getGuestType() == GuestType.Shared) {
// remove the dhcpservice ip if this is the last nic in subnet.
DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
if (dhcpServiceProvider != null
isDhcpAccrossMultipleSubnetsSupported(dhcpServiceProvider)
isLastNicInSubnet(nic)) {
removeDhcpServiceInSubnet(nic);
}
}
Daan, it would still break if network does not have DHCP service enabled,
best would be to check
'_networkModel.areServicesSupportedInNetwork(network.getId(),
Service.Dhcp)' then get the provider.
On Fri, Feb 7, 2014 at 6:55 AM, Daan Hoogland daan.hoogl...@gmail.com
wrote:
second thought,
Soheils mail bounces and the commit does not refer a ticket from jira.
I am going to revert. I should have been more vigilant. sorry.
On Fri, Feb 7, 2014 at 6:49 AM, Daan Hoogland daan.hoogl...@gmail.com
wrote:
will do Alena,
thanks for the headsup
On Thu, Feb 6, 2014 at 10:42 PM, Alena Prokharchyk
alena.prokharc...@citrix.com wrote:
Soheil/Daan,
The commit in the subject breaks network System vms destroy (VR, SSVM,
CPVM), resulting in the network removal failures. Following line
replacement
causes the failure:
- if (vm.getType() == Type.User
isDhcpAccrossMultipleSubnetsSupported(network)
isLastNicInSubnet(nic)
network.getTrafficType() == TrafficType.Guest
With
+DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
When you try to call getDhcpServiceProvider(network), it throws an
exception
because DHCP service is not enabled in Public/Control networks of
system vms
nics. So system vm always fails to expunge.
Could you please fix it by checking if DHCP service is enabled on the
network, before getting the DHCP service provider?
Thanks,
Alena.
--
Daan
--
Daan
--
Daan
--
Daan
Review Request 17841: CLOUDSTACK-5959: Fixed cascading failures in portable IP test cases
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17841/ --- Review request for cloudstack and Girish Shilamkar. Bugs: CLOUDSTACK-5959 https://issues.apache.org/jira/browse/CLOUDSTACK-5959 Repository: cloudstack-git Description --- Failure in firewall creation is leading to incomplete cleanup and eventually cascading failure of the test cases. Made changes so that even in case of failure in one test case, other test cases are not affected and cleanup is performed gracefully. Also did 1) Code cleanup 2) Modifying imports 3) Adding FS and test plan links 4) Refactoring code Diffs - test/integration/component/test_portable_ip.py d397ec4 tools/marvin/marvin/integration/lib/common.py 550de1a Diff: https://reviews.apache.org/r/17841/diff/ Testing --- Tested on VMware Thanks, Gaurav Aradhye
Re: [PROPOSAL] load balancing configuration for VM nic secondary ips
I discussed with Antonio internally; we are in diagreement. I would like to see some kind of context diagram inside java class javadocs. Antonio would go for a seperate file (on package level, Antonio?) any others have brain waves on this? On Thu, Feb 6, 2014 at 2:11 PM, Jayapal Reddy Uradi jayapalreddy.ur...@citrix.com wrote: Hi, Multiple ips per nic feature allows user to acquire more ip address on vm nic. On these ip addresses user able to configure PF and static NAT. With this feature user can also configure the load balancing rules for vm secondary ips also. Please review the below and give your comments. https://cwiki.apache.org/confluence/display/CLOUDSTACK/Configuring+load+balancing+rules+for+VM+nic+secondary+ips Ticket id: CLOUDSTACK-2692 Thanks, Jayapal -- Daan
Re: [PROPOSAL] load balancing configuration for VM nic secondary ips
ignore, i will resend in the right thread On Fri, Feb 7, 2014 at 11:58 AM, Daan Hoogland daan.hoogl...@gmail.com wrote: I discussed with Antonio internally; we are in diagreement. I would like to see some kind of context diagram inside java class javadocs. Antonio would go for a seperate file (on package level, Antonio?) any others have brain waves on this? On Thu, Feb 6, 2014 at 2:11 PM, Jayapal Reddy Uradi jayapalreddy.ur...@citrix.com wrote: Hi, Multiple ips per nic feature allows user to acquire more ip address on vm nic. On these ip addresses user able to configure PF and static NAT. With this feature user can also configure the load balancing rules for vm secondary ips also. Please review the below and give your comments. https://cwiki.apache.org/confluence/display/CLOUDSTACK/Configuring+load+balancing+rules+for+VM+nic+secondary+ips Ticket id: CLOUDSTACK-2692 Thanks, Jayapal -- Daan -- Daan
Re: PlantUML
I discussed with Antonio internally; we are in diagreement. I would like to see some kind of context diagram inside java class javadocs. Antonio would go for a seperate file (on package level, Antonio?) any others have brain waves on this? On Thu, Feb 6, 2014 at 2:03 PM, Antonio Fornié Casarrubios antonio.for...@gmail.com wrote: Hi all, In order to make it easier for developers to understand better cloudstack and its code, I think sometimes an UML diagram could help a lot. IMO much more than comments and headers most times. My favorite tool for UML is PlantUML mainly because it's plain text that can be viewed with a Plant plugin (or any other PlantUML flavour). So if a diagram gets obsolete it's easy and fast to update the PlantUML file, commit it and everybody will see the diagram updated. that way we don't commit images or strange formats, just text. And if considered necessary any of these diagrams can also be exported (for example with the same PlantUML eclipse plugin). If you guys agree, I would then proceed and commit my first PlantUML files. By the way, in such case I would also like confirmation o where to place them for each module. http://plantuml.sourceforge.net/ Thanks and cheers Antonio Fornie Schuberg Philis - MCE -- Daan
systemvm-4.3.0.iso
Have any changes been made to the deployment of the systemvm-4.3.0.iso between RC2 and RC3 ?? I can't get a system VMs to fully start in hyperv or xenserver - all issues seem to relate to the systemvm-4.3.0.iso - which doesn't seem to be making across to primary storage in hyper-v or properly configuring the secondary storage VM under xenserver... Regards Paul Angus Senior Consultant / Cloud Architect [cid:image002.png@01CE1071.C6CC9C10] S: +44 20 3603 0540tel:+442036030540 | M: +4tel:+44796816158147711418784 | T: @CloudyAngus paul.an...@shapeblue.commailto:paul.an...@shapeblue.com | www.shapeblue.comhttp://www.shapeblue.com/ | Twitter:@shapebluehttps://twitter.com/ ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N 4HS Need Enterprise Grade Support for Apache CloudStack? Our CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ offers the best 24/7 SLA for CloudStack Environments. Apache CloudStack Bootcamp training courses **NEW!** CloudStack 4.2.1 traininghttp://shapeblue.com/cloudstack-training/ 18th-19th February 2014, Brazil. Classroomhttp://shapeblue.com/cloudstack-training/ 17th-23rd March 2014, Region A. Instructor led, On-linehttp://shapeblue.com/cloudstack-training/ 24th-28th March 2014, Region B. Instructor led, On-linehttp://shapeblue.com/cloudstack-training/ 16th-20th June 2014, Region A. Instructor led, On-linehttp://shapeblue.com/cloudstack-training/ 23rd-27th June 2014, Region B. Instructor led, On-linehttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: commit 6523c068695d0431070060667c222eb40d54b14d breaks network removal
Hope this is not blocking at the moment so I can take the time to add
a unit test. If not I have the code ready to ship. please bug me.
On Fri, Feb 7, 2014 at 11:32 AM, Daan Hoogland daan.hoogl...@gmail.com wrote:
thanks Murali, will do
On Fri, Feb 7, 2014 at 9:58 AM, murali reddy muralimmre...@gmail.com wrote:
On Fri, Feb 7, 2014 at 11:34 AM, Daan Hoogland
daan.hoogl...@gmail.comwrote:
Alena,
The revert didn't apply. Would the folowing do the trick?
if (vm.getType() == Type.User
network.getTrafficType() == TrafficType.Guest
network.getGuestType() == GuestType.Shared) {
// remove the dhcpservice ip if this is the last nic in subnet.
DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
if (dhcpServiceProvider != null
isDhcpAccrossMultipleSubnetsSupported(dhcpServiceProvider)
isLastNicInSubnet(nic)) {
removeDhcpServiceInSubnet(nic);
}
}
Daan, it would still break if network does not have DHCP service enabled,
best would be to check
'_networkModel.areServicesSupportedInNetwork(network.getId(),
Service.Dhcp)' then get the provider.
On Fri, Feb 7, 2014 at 6:55 AM, Daan Hoogland daan.hoogl...@gmail.com
wrote:
second thought,
Soheils mail bounces and the commit does not refer a ticket from jira.
I am going to revert. I should have been more vigilant. sorry.
On Fri, Feb 7, 2014 at 6:49 AM, Daan Hoogland daan.hoogl...@gmail.com
wrote:
will do Alena,
thanks for the headsup
On Thu, Feb 6, 2014 at 10:42 PM, Alena Prokharchyk
alena.prokharc...@citrix.com wrote:
Soheil/Daan,
The commit in the subject breaks network System vms destroy (VR, SSVM,
CPVM), resulting in the network removal failures. Following line
replacement
causes the failure:
- if (vm.getType() == Type.User
isDhcpAccrossMultipleSubnetsSupported(network)
isLastNicInSubnet(nic)
network.getTrafficType() == TrafficType.Guest
With
+DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
When you try to call getDhcpServiceProvider(network), it throws an
exception
because DHCP service is not enabled in Public/Control networks of
system vms
nics. So system vm always fails to expunge.
Could you please fix it by checking if DHCP service is enabled on the
network, before getting the DHCP service provider?
Thanks,
Alena.
--
Daan
--
Daan
--
Daan
--
Daan
--
Daan
RE: systemvm-4.3.0.iso
Hi Paul, For hyper-v, if older systemvm iso is present with the same name then it'll not get updated. You can remove the systemvm directory from secondary storage. Also on you hyper-v host, systemvm.iso gets put on RootDrive:\Users\Public\Documents\Hyper-V directory. You can delete it. When a system vm (ssvm or consolve proxy or router vm) is started, it'll get copied over. Regards, Devdeep From: Paul Angus [mailto:paul.an...@shapeblue.com] Sent: Friday, February 7, 2014 4:35 PM To: dev@cloudstack.apache.org Subject: systemvm-4.3.0.iso Have any changes been made to the deployment of the systemvm-4.3.0.iso between RC2 and RC3 ?? I can't get a system VMs to fully start in hyperv or xenserver - all issues seem to relate to the systemvm-4.3.0.iso - which doesn't seem to be making across to primary storage in hyper-v or properly configuring the secondary storage VM under xenserver... Regards Paul Angus Senior Consultant / Cloud Architect [cid:image002.png@01CE1071.C6CC9C10] S: +44 20 3603 0540tel:+442036030540 | M: +4tel:+44796816158147711418784 | T: @CloudyAngus paul.an...@shapeblue.commailto:paul.an...@shapeblue.com | www.shapeblue.comhttp://www.shapeblue.com/ | Twitter:@shapebluehttps://twitter.com/ ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N 4HS Need Enterprise Grade Support for Apache CloudStack? Our CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ offers the best 24/7 SLA for CloudStack Environments. Apache CloudStack Bootcamp training courses **NEW!** CloudStack 4.2.1 traininghttp://shapeblue.com/cloudstack-training/ 18th-19th February 2014, Brazil. Classroomhttp://shapeblue.com/cloudstack-training/ 17th-23rd March 2014, Region A. Instructor led, On-linehttp://shapeblue.com/cloudstack-training/ 24th-28th March 2014, Region B. Instructor led, On-linehttp://shapeblue.com/cloudstack-training/ 16th-20th June 2014, Region A. Instructor led, On-linehttp://shapeblue.com/cloudstack-training/ 23rd-27th June 2014, Region B. Instructor led, On-linehttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17747/ --- (Updated Feb. 7, 2014, 11:19 a.m.) Review request for cloudstack, Abhinandan Prateek, Kishan Kavala, and Nitin Mehta. Bugs: CLOUDSTACK-6023 https://issues.apache.org/jira/browse/CLOUDSTACK-6023 Repository: cloudstack-git Description (updated) --- CLOUDSTACK-6023: Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits Changed the VCPU max limit to 16 and provided a cluster level configuration parameter for this max limit named xen.vm.vcpu.max Diffs (updated) - api/src/com/cloud/agent/api/to/VirtualMachineTO.java e6240ff plugins/hypervisors/xen/src/com/cloud/hypervisor/XenServerGuru.java 28bd724 plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java 48ccf70 Diff: https://reviews.apache.org/r/17747/diff/ Testing --- Thanks, Harikrishna Patnala
Review Request 17842: fixed vmsync for hyperv
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17842/ --- Review request for cloudstack, Devdeep Singh and Rajesh Battala. Bugs: CLOUDSTACK-6054 https://issues.apache.org/jira/browse/CLOUDSTACK-6054 Repository: cloudstack-git Description --- fixed vmsync for hyperv. This fix adds an api call which return the hostvmstatereport and made appropriate changes to HypervDirectConnectResource.java Diffs - plugins/hypervisors/hyperv/DotNet/ServerResource/HypervResource/CloudStackTypes.cs 9637994 plugins/hypervisors/hyperv/DotNet/ServerResource/HypervResource/HypervResourceController.cs a132606 plugins/hypervisors/hyperv/DotNet/ServerResource/HypervResource/IWmiCallsV2.cs 5f814c5 plugins/hypervisors/hyperv/DotNet/ServerResource/HypervResource/WmiCallsV2.cs f5a04dc plugins/hypervisors/hyperv/src/com/cloud/hypervisor/hyperv/resource/HypervDirectConnectResource.java a4dca6a Diff: https://reviews.apache.org/r/17842/diff/ Testing --- Performed basic vm operations externally and verifying their state got updated correctly in cloudstack Thanks, Anshul Gangwar
Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17747/#review33924 --- Ship it! Ship It! - Kishan Kavala On Feb. 7, 2014, 4:49 p.m., Harikrishna Patnala wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17747/ --- (Updated Feb. 7, 2014, 4:49 p.m.) Review request for cloudstack, Abhinandan Prateek, Kishan Kavala, and Nitin Mehta. Bugs: CLOUDSTACK-6023 https://issues.apache.org/jira/browse/CLOUDSTACK-6023 Repository: cloudstack-git Description --- CLOUDSTACK-6023: Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits Changed the VCPU max limit to 16 and provided a cluster level configuration parameter for this max limit named xen.vm.vcpu.max Diffs - api/src/com/cloud/agent/api/to/VirtualMachineTO.java e6240ff plugins/hypervisors/xen/src/com/cloud/hypervisor/XenServerGuru.java 28bd724 plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java 48ccf70 Diff: https://reviews.apache.org/r/17747/diff/ Testing --- Thanks, Harikrishna Patnala
RE: systemvm-4.3.0.iso
Thanks for the info Devdeep,
so yes the systemvm iso is getting copied to the local disk, but I'm getting
the following error:
ERROR HypervResource.HypervResourceController
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - com.cloud.agent.api.StartCommand fail
on exceptionObject reference not set to an instance of an object.
System.NullReferenceException: Object reference not set to an instance of an
object.
at HypervResource.WmiCallsV2.DeployVirtualMachine(Object jsonObj, String
systemVmIso)
at HypervResource.HypervResourceController.StartCommand(Object cmd)
agent log:
2014-02-07 11:19:56,685 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Going ahead with create VM s-27-VM, 1
vcpus, 512MB RAM
2014-02-07 11:19:56,716 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Created VM s-27-VM
2014-02-07 11:19:57,075 [28] INFO HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - VM with display name s-27-VM has GUID
9E6D3606-8210-4E84-8796-BEB7970785F4
2014-02-07 11:19:57,075 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Resources for vm s-27-VM: 512 MB
memory, 1 vcpus
2014-02-07 11:19:57,106 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - New controller type
Microsoft:Hyper-V:Synthetic SCSI Controller WMI path is
\\WIN-G23HSIAIU4O\root\virtualization\v2:Msvm_ResourceAllocationSettingData.InstanceID=Microsoft:9E6D3606-8210-4E84-8796-BEB7970785F4\\57035109-ED28-4F65-8BCB-487CFC66F8E8\\0s
2014-02-07 11:19:57,106 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Going to create s-27-VM with attached
voluem ROOT-27 at \\10.0.1.27\hypervPri\stor\Primary\ROOT-27.vhd
2014-02-07 11:19:57,106 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Create disk type
Microsoft:Hyper-V:Synthetic Disk Drive (Named: ROOT-27), on vm s-27-VM ,
inserting disk\\10.0.1.27\hypervPri\stor\Primary\ROOT-27.vhd
2014-02-07 11:19:57,106 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Creating DISK for VM s-27-VM (GUID
9E6D3606-8210-4E84-8796-BEB7970785F4) by attaching
\\10.0.1.27\hypervPri\stor\Primary\ROOT-27.vhd
2014-02-07 11:19:57,403 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Creating disk drive type
Microsoft:Hyper-V:Synthetic Disk Drive, parent IDE controller is
\\WIN-G23HSIAIU4O\root\virtualization\v2:Msvm_ResourceAllocationSettingData.InstanceID=Microsoft:9E6D3606-8210-4E84-8796-BEB7970785F4\\83F8638B-8DCA-4152-9EDA-2CA8B33039B4\\0
and address on controller is 0
2014-02-07 11:19:57,418 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - New disk drive type
Microsoft:Hyper-V:Synthetic Disk Drive WMI path is
\\WIN-G23HSIAIU4O\root\virtualization\v2:Msvm_ResourceAllocationSettingData.InstanceID=Microsoft:9E6D3606-8210-4E84-8796-BEB7970785F4\\83F8638B-8DCA-4152-9EDA-2CA8B33039B4\\0\\0\\Ds
2014-02-07 11:19:57,574 [28] INFO HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Created disk
\\WIN-G23HSIAIU4O\root\virtualization\v2:Msvm_StorageAllocationSettingData.InstanceID=Microsoft:9E6D3606-8210-4E84-8796-BEB7970785F4\\83F8638B-8DCA-4152-9EDA-2CA8B33039B4\\0\\0\\L
for VM s-27-VM (GUID 9E6D3606-8210-4E84-8796-BEB7970785F4), image
\\10.0.1.27\hypervPri\stor\Primary\ROOT-27.vhd
2014-02-07 11:19:57,574 [28] ERROR HypervResource.HypervResourceController
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - com.cloud.agent.api.StartCommand fail
on exceptionObject reference not set to an instance of an object.
System.NullReferenceException: Object reference not set to an instance of an
object.
at HypervResource.WmiCallsV2.DeployVirtualMachine(Object jsonObj, String
systemVmIso)
at HypervResource.HypervResourceController.StartCommand(Object cmd)
2014-02-07 11:19:57,574 [28] INFO HypervResource.HypervResourceController
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - {
com.cloud.agent.api.StartAnswer: {
result: false,
details: com.cloud.agent.api.StartCommand fail on exceptionObject
reference not set to an instance of an object.,
vm: {
id: 27,
name: s-27-VM,
type: SecondaryStorageVm,
cpus: 1,
minSpeed: 125,
maxSpeed: 500,
minRam: 536870912,
maxRam: 536870912,
arch: x86_64,
os: Debian GNU/Linux 5.0 (64-bit),
bootArgs: template=domP type=secstorage host=10.0.100.4 port=8250
name=s-27-VM zone=1 pod=1 guid=s-27-VM
resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
instance=SecStorage sslcopy=true role=templateProcessor mtu=1500
eth2ip=10.0.106.1 eth2mask=255.255.0.0 gateway=10.0.1.1 eth0ip=169.254.0.63
eth0mask=255.255.0.0 eth1ip=10.0.106.248 eth1mask=255.255.0.0
mgmtcidr=10.0.0.0/16 localgw=10.0.1.1 private.network.device=eth1
eth3ip=10.0.106.249 eth3mask=255.255.0.0 storageip=10.0.106.249
storagenetmask=255.255.0.0
Portable IP - Network rules failure
Hi all, Firewall rule creation and Enabling static NAT rule for Portable public IP are failing, and is observed in daily build too. It was masked by other cascading failures for which I have already added patch ( https://reviews.apache.org/r/17841/). I have observed these issues on VMware yet and logged two bugs for this. Please pick up/update/reply if you can. 1) Firewall rule failure - CLOUDSTACK-6055https://issues.apache.org/jira/browse/CLOUDSTACK-6055 2) Static NAT rule failure - CLOUDSTACK-6056https://issues.apache.org/jira/browse/CLOUDSTACK-6056 Management server logs are attached to the respective bugs. Regards, Gaurav
RE: Removing deploy\load options from marvinplugin
I believe we are deviating with too many notes here. Lets put things in perspective. 1. Initial point was to understand and take inputs to have and work with marvinplugin using less options for running, minimize options we have currently and can we remove few and work with them? and i believe that's what you mentioned to have less options in earlier mail. 2. In the initial mail, it was mentioned that if there is a change, it will effect few areas like devcloud\simulator, provided if there is a change, starting this thread is to know a point of view and see the impact, that's what is to have clarified here. I see there is no impact there in other areas mentioned, that does not mean we are agreeing for a change. 3. Export\Import configuration from marvin\cloudstack is a separate issue for discussion, i believe you can include it in a separate thread for now and discuss there. People can have their say of having this facility or not. Regarding its a leak or not that's a separate discussion to have and how to design or implement again, that's nothing to do with options change we mentioned. This will keep the current discussions easier to follow. 4. deploy VS load, in the earlier mail, i didn't mentioned to remove deploy, i said only load option. Lets see what load option is doing currently, It does the below, which i believe can still be possible with one deploy option. Here, we are creating a client with configuration provided. This is happening even with load option and as well as inside of deploy option . I believe we can control this behavior with single deploy option. If deploy option is not provided, then it works as though load option else deploy option of currently. Please let me know where updating the global configuration is happening as part of current loadCfg option? def loadCfg(self): try: self.config = configGenerator.getSetupConfig(self.configFile) except: raise cloudstackException.InvalidParameterException( Failed to load config %s % self.configFile) ''' Retrieving Management Server Connection Details ''' mgtDetails = self.config.mgtSvr[0] ''' Retrieving Database Connection Details''' dbSvrDetails = self.config.dbSvr loggers = self.config.logger testClientLogFile = None self.testCaseLogFile = None self.testResultLogFile = None if loggers is not None and len(loggers) 0: for log in loggers: if log.name == TestClient: testClientLogFile = log.file elif log.name == TestCase: self.testCaseLogFile = log.file elif log.name == TestResult: self.testResultLogFile = log.file testClientLogger = None if testClientLogFile is not None: testClientLogger = logging.getLogger(testclient.testengine.run) fh = logging.FileHandler(testClientLogFile) fh.setFormatter(logging.Formatter( %(asctime)s - %(levelname)s - %(name)s\ - %(message)s) ) testClientLogger.addHandler(fh) testClientLogger.setLevel(logging.INFO) self.testClientLogger = testClientLogger self.testClient = \ cloudstackTestClient.\ cloudstackTestClient(mgtDetails, dbSvrDetails, logging=self.testClientLogger) logger=self.tcRunLogger) if mgtDetails.apiKey is None: mgtDetails.apiKey, mgtDetails.securityKey = self.registerApiKey()there run a deployDC with configuration provided and if not 5. Also, its better if know where we are upading the other global configuration you mentioned as part of load option? Here, its just creating the client based upon configuration provided. 6. why deploying cloudstack is part of nose tests now and where we mentioned it is and make it a 4 step process? We are anyways not doing it now as part of nosetests. We are adding one more addition of restart CS, which is totally not required as part of nosetets. Iam not sure adding a restart simplifies and makes it little more complex. 1. deploy cloudstack 2. deploydatacenter (done using nose earlier) 3. restart cloudstack 4. run tests (also done by nose earlier) 7. The reason for separation is to keep things simple. As a user, i can run below. The reason i mentioned to separate deploy out of nose tests is we are not doing anything as such to report a failure for bvt\regression etc for deployDC, we just exit without checks for deployDC. Also, i mentioned the current flow as an eample, where we are using explicit deployDC, followed by tests using nose plugin. If we have references where we are using it in both flows then we can think of it, let us know? 1. DeployDC 2. Run test cases with nose tests. 8. The logging issue: What i
RE: Not able to add primary Storage Cloudstack 4.3
Tejas, Check out the dev list I've just posted an issue with the system vms not starting - you might be having the same issue. check the agent log for: ERROR HypervResource.HypervResourceController [6b05182a-efa1-4105-bc5c-0a49c55674f0] - com.cloud.agent.api.StartCommand fail on exceptionObject reference not set to an instance of an object. System.NullReferenceException: Object reference not set to an instance of an object. at HypervResource.WmiCallsV2.DeployVirtualMachine(Object jsonObj, String systemVmIso) at HypervResource.HypervResourceController.StartCommand(Object cmd) Regards, Paul Angus Cloud Architect S: +44 20 3603 0540 | M: +447711418784 | T: @CloudyAngus paul.an...@shapeblue.com -Original Message- From: Tejas Gadaria [mailto:refond.g...@gmail.com] Sent: 07 February 2014 11:31 To: Rajesh Battala Cc: us...@cloudstack.apache.org; dev@cloudstack.apache.org Subject: Re: Not able to add primary Storage Cloudstack 4.3 Hi Rajesh, Thanks for providing API. I have successfully attached primary storage to cloudstack. I am facing problem with SSVM and CPVM also, probably I will initiate new mail-trail ssytem vm issues. Reards, Tejas On Thu, Feb 6, 2014 at 12:27 PM, Rajesh Battala rajesh.batt...@citrix.comwrote: Tejas, This is how the api command looks like to create primary and secondary storage. command=createStoragePoolzoneid=581bc2b6-0295-40e0-90ed-2717e239c2b6 podId=896cff2e-d51e-454b-97ed-cab078bf3be2clusterid=3785c58a-d891-448 2-9e6f-72f2060b80d3name=psscope=clusterurl=cifs%3A%2F%2F10.102.192. 150%2Fsmb-share%2Frajesh%2Fprimary%3Fuser%3DAdministrator%26domain%3DB LRresponse=jsonsessionkey=1L5raRYhxHCAgXHJ1WGvLjs7Zws%3D_=139166689 6777 command=addImageStoreresponse=jsonsessionkey=1L5raRYhxHCAgXHJ1WGvLjs 7Zws%3Dname=secprovider=SMBzoneid=581bc2b6-0295-40e0-90ed-2717e239c 2b6url=cifs%3A%2F%2F10.102.192.150%2Fsmb-share%2Frajesh%2Fsecondary%3 Fuser%3DAdministrator%26domain%3DBLR_=1391666908684 in the wizard specify the path as /smb-share/rajesh/primary Similar for secondary. Provide credentials and domain name in the wizard. Make sure you were able to access the share from the hyperv hosts before adding them to CS. Thanks Rajesh Battala *From:* Tejas Gadaria [mailto:refond.g...@gmail.com] *Sent:* Thursday, February 6, 2014 11:03 AM *To:* us...@cloudstack.apache.org *Cc:* Rajesh Battala; dev@cloudstack.apache.org *Subject:* Re: Not able to add primary Storage Cloudstack 4.3 Hi, I am not able to attach primary storage in Cloudstack as cifs. So what I am doing wrong OR what are the issues with primary storage.? Regards Tejas On Wed, Feb 5, 2014 at 4:35 PM, Tejas Gadaria refond.g...@gmail.com wrote: Hi Paul, forgot to mention, hyper-v agent is running. Regards, Tejas On Wed, Feb 5, 2014 at 4:33 PM, Tejas Gadaria refond.g...@gmail.com wrote: Hi Paul, yesterday when I was trying to add secondary storage with /hcloud/secondary , In logs I was not able to see much activity, but storage got attached. I have uploaded screenshot in previous mail. I have reattached the secondary storage, and it logs are Trying to add a new data store at cifs:// 10.129.151.55/hcloud/secondary?user=administratordomain=nwx.com to data center 3 2014-02-05 15:36:07,825 DEBUG [c.c.u.UriUtils] (catalina-exec-20:ctx-a19af0fa ctx-fb83ed9b) foundUser istrue 2014-02-05 15:36:07,825 DEBUG [c.c.u.UriUtils] (catalina-exec-20:ctx-a19af0fa ctx-fb83ed9b) foundPswd istrue I have not enable the zone yet. In attached screenshot of secondary storage in previous mail protocol:cifs provider : NFS I suspect, it because I have taken chunk of NFS datastore, mounted and share it via Windows fie server (Standalone) please find logs in attached file. Regards, Tejas On Tue, Feb 4, 2014 at 5:52 PM, Paul Angus paul.an...@shapeblue.com wrote: Hey Tejas, cifs%3A%2F%2F10.129.151.155hcloud%2Fsecondary%3Fuser%3Dadministrator%2 6domain%3Dnwx.com Theres a / (%2F) missing between 10.129.151.155 and hcloud - my recollection is that primary and secondary storage UIs are not consistent, add the secondary path as /hcloud/secondary re primary - is the agent service running? it doesn't seem to (always) automatically start even when it's set to automatic in services. Regards, Paul Angus Cloud Architect S: +44 20 3603 0540 | M: +447711418784 | T: @CloudyAngus paul.an...@shapeblue.com From: Tejas Gadaria [mailto:refond.g...@gmail.com] Sent: 04 February 2014 12:15 To: us...@cloudstack.apache.org Cc: rajesh.batt...@citrix.com; dev@cloudstack.apache.org Subject: Re: Not able to add primary Storage Cloudstack 4.3 Hi Paul, I tried with hcloud/primary and primary storage shows Unsupported command issued. secondary storage shows Failed to add data store: cifs URI missing host and/or path. Make sure it's of the format cifs://hostname/path?user=usernamepassword=password After that
Source build failing in DevCloud
Hi, This is my first attempt at building CloudStack from the source code and decided on using DevCloud. I followed the wiki for using DevCloud https://cwiki.apache.org/confluence/display/CLOUDSTACK/DevCloud. I want to build the 4.3 release and used the updated DevCloud 2.0 appliance and as per the instructions. I git cloned the latest master code, but when I then run mvn -P developer,systemvm clean install. I get a compilation/build failure (see output below). Q1) Have I missed something and why I am I getting this error? Q2) I see it is building 4.4.0-SNAPSHOT how do I get it to build 4.3? Thanks. [INFO] [INFO] Building Apache CloudStack Developer Tools - Checkstyle Configuration 4.4.0-SNAPSHOT [INFO] [INFO] [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ checkstyle --- [INFO] Deleting /opt/cloudstack/cloudstack/tools/checkstyle/target [INFO] [INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ checkstyle --- [debug] execute contextualize [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] Copying 1 resource [INFO] [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ checkstyle --- [INFO] No sources to compile [INFO] [INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ checkstyle --- [debug] execute contextualize [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory /opt/cloudstack/cloudstack/tools/checkstyle/src/test/resources [INFO] [INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ checkstyle --- [INFO] No sources to compile [INFO] [INFO] --- maven-surefire-plugin:2.10:test (default-test) @ checkstyle --- [INFO] No tests to run. [INFO] Surefire report directory: /opt/cloudstack/cloudstack/tools/checkstyle/target/surefire-reports --- T E S T S --- Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] --- maven-jar-plugin:2.3.2:jar (default-jar) @ checkstyle --- [INFO] Building jar: /opt/cloudstack/cloudstack/tools/checkstyle/target/checkstyle-4.4.0-SNAPSHOT.jar [INFO] [INFO] --- maven-install-plugin:2.3.1:install (default-install) @ checkstyle --- [INFO] Installing /opt/cloudstack/cloudstack/tools/checkstyle/target/checkstyle-4.4.0-SNAPSHOT.jar to /root/.m2/repository/org/apache/cloudstack/checkstyle/4.4.0-SNAPSHOT/checkstyle-4.4.0-SNAPSHOT.jar [INFO] Installing /opt/cloudstack/cloudstack/tools/checkstyle/pom.xml to /root/.m2/repository/org/apache/cloudstack/checkstyle/4.4.0-SNAPSHOT/checkstyle-4.4.0-SNAPSHOT.pom [INFO] [INFO] [INFO] Building Apache CloudStack 4.4.0-SNAPSHOT [INFO] [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloudstack --- [INFO] Deleting /opt/cloudstack/cloudstack/target (includes = [**/*], excludes = []) [INFO] Deleting /opt/cloudstack/cloudstack (includes = [target, dist], excludes = []) [INFO] [INFO] --- maven-remote-resources-plugin:1.3:process (default) @ cloudstack --- [INFO] [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ cloudstack --- [INFO] [INFO] --- maven-install-plugin:2.3.1:install (default-install) @ cloudstack --- [INFO] Installing /opt/cloudstack/cloudstack/pom.xml to /root/.m2/repository/org/apache/cloudstack/cloudstack/4.4.0-SNAPSHOT/cloudstack-4.4.0-SNAPSHOT.pom [INFO] [INFO] [INFO] Building Apache CloudStack Maven Conventions Parent 4.4.0-SNAPSHOT [INFO] [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-maven-standard --- [INFO] Deleting /opt/cloudstack/cloudstack/maven-standard/target (includes = [**/*], excludes = []) [INFO] Deleting /opt/cloudstack/cloudstack/maven-standard (includes = [target, dist], excludes = []) [INFO] [INFO] --- maven-checkstyle-plugin:2.11:check (cloudstack-checkstyle) @ cloud-maven-standard --- [INFO] Starting audit... Audit done. [INFO] [INFO] --- maven-remote-resources-plugin:1.3:process (default) @ cloud-maven-standard --- [INFO] [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ cloud-maven-standard --- [INFO] [INFO] --- maven-install-plugin:2.3.1:install (default-install) @ cloud-maven-standard --- [INFO] Installing /opt/cloudstack/cloudstack/maven-standard/pom.xml to /root/.m2/repository/org/apache/cloudstack/cloud-maven-standard/4.4.0-SNAPSHOT/cloud-maven-standard-4.4.0-SNAPSHOT.pom [INFO] [INFO]
Re: Removing deploy\load options from marvinplugin
tl;dr i think your point 4 clarifies your change. remove --load and only run tests unless --deploy is given. in the latter - do a deploy. On Fri, Feb 07, 2014 at 11:30:19AM +, Santhosh Edukulla wrote: I believe we are deviating with too many notes here. Lets put things in perspective. 1. Initial point was to understand and take inputs to have and work with marvinplugin using less options for running, minimize options we have currently and can we remove few and work with them? and i believe that's what you mentioned to have less options in earlier mail. 2. In the initial mail, it was mentioned that if there is a change, it will effect few areas like devcloud\simulator, provided if there is a change, starting this thread is to know a point of view and see the impact, that's what is to have clarified here. I see there is no impact there in other areas mentioned, that does not mean we are agreeing for a change. 3. Export\Import configuration from marvin\cloudstack is a separate issue for discussion, i believe you can include it in a separate thread for now and discuss there. People can have their say of having this facility or not. Regarding its a leak or not that's a separate discussion to have and how to design or implement again, that's nothing to do with options change we mentioned. This will keep the current discussions easier to follow. 4. deploy VS load, in the earlier mail, i didn't mentioned to remove deploy, i said only load option. Lets see what load option is doing currently, It does the below, which i believe can still be possible with one deploy option. Here, we are creating a client with configuration provided. This is happening even with load option and as well as inside of deploy option . I believe we can control this behavior with single deploy option. If deploy option is not provided, then it works as though load option else deploy option of currently. Please let me know where updating the global configuration is happening as part of current loadCfg option? def loadCfg(self): try: self.config = configGenerator.getSetupConfig(self.configFile) except: raise cloudstackException.InvalidParameterException( Failed to load config %s % self.configFile) ''' Retrieving Management Server Connection Details ''' mgtDetails = self.config.mgtSvr[0] ''' Retrieving Database Connection Details''' dbSvrDetails = self.config.dbSvr loggers = self.config.logger testClientLogFile = None self.testCaseLogFile = None self.testResultLogFile = None if loggers is not None and len(loggers) 0: for log in loggers: if log.name == TestClient: testClientLogFile = log.file elif log.name == TestCase: self.testCaseLogFile = log.file elif log.name == TestResult: self.testResultLogFile = log.file testClientLogger = None if testClientLogFile is not None: testClientLogger = logging.getLogger(testclient.testengine.run) fh = logging.FileHandler(testClientLogFile) fh.setFormatter(logging.Formatter( %(asctime)s - %(levelname)s - %(name)s\ - %(message)s) ) testClientLogger.addHandler(fh) testClientLogger.setLevel(logging.INFO) self.testClientLogger = testClientLogger self.testClient = \ cloudstackTestClient.\ cloudstackTestClient(mgtDetails, dbSvrDetails, logging=self.testClientLogger) logger=self.tcRunLogger) if mgtDetails.apiKey is None: mgtDetails.apiKey, mgtDetails.securityKey = self.registerApiKey()there run a deployDC with configuration provided and if not 5. Also, its better if know where we are upading the other global configuration you mentioned as part of load option? Here, its just creating the client based upon configuration provided. 6. why deploying cloudstack is part of nose tests now and where we mentioned it is and make it a 4 step process? We are anyways not doing it now as part of nosetests. We are adding one more addition of restart CS, which is totally not required as part of nosetets. Iam not sure adding a restart simplifies and makes it little more complex. 1. deploy cloudstack 2. deploydatacenter (done using nose earlier) 3. restart cloudstack 4. run tests (also done by nose earlier) 7. The reason for separation is to keep things simple. As a user, i can run below. The reason i mentioned to separate deploy out of nose tests is we are not doing anything as such to report a failure for bvt\regression etc for deployDC, we just exit without checks for
RE: Source build failing in DevCloud
I've had a quick look at the error statements - what version of Java is on the devcloud vm? Also, if you checkout the 4.3 branch, git pull then compile you should get the 4.3 current build. Regards Alex Hitchins D: +44 1892 523 587 | S: +44 20 3603 0540 | M: +44 7788 423 969 alex.hitch...@shapeblue.com -Original Message- From: CK [mailto:cloudw...@gmail.com] Sent: 07 February 2014 11:41 To: dev@cloudstack.apache.org Subject: Source build failing in DevCloud Hi, This is my first attempt at building CloudStack from the source code and decided on using DevCloud. I followed the wiki for using DevCloud https://cwiki.apache.org/confluence/display/CLOUDSTACK/DevCloud. I want to build the 4.3 release and used the updated DevCloud 2.0 appliance and as per the instructions. I git cloned the latest master code, but when I then run mvn -P developer,systemvm clean install. I get a compilation/build failure (see output below). Q1) Have I missed something and why I am I getting this error? Q2) I see it is building 4.4.0-SNAPSHOT how do I get it to build 4.3? Thanks. [INFO] [INFO] Building Apache CloudStack Developer Tools - Checkstyle Configuration 4.4.0-SNAPSHOT [INFO] [INFO] [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ checkstyle --- [INFO] Deleting /opt/cloudstack/cloudstack/tools/checkstyle/target [INFO] [INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ checkstyle --- [debug] execute contextualize [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] Copying 1 resource [INFO] [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ checkstyle --- [INFO] No sources to compile [INFO] [INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ checkstyle --- [debug] execute contextualize [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory /opt/cloudstack/cloudstack/tools/checkstyle/src/test/resources [INFO] [INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ checkstyle --- [INFO] No sources to compile [INFO] [INFO] --- maven-surefire-plugin:2.10:test (default-test) @ checkstyle --- [INFO] No tests to run. [INFO] Surefire report directory: /opt/cloudstack/cloudstack/tools/checkstyle/target/surefire-reports --- T E S T S --- Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] --- maven-jar-plugin:2.3.2:jar (default-jar) @ checkstyle --- [INFO] Building jar: /opt/cloudstack/cloudstack/tools/checkstyle/target/checkstyle-4.4.0-SNAPSHOT.jar [INFO] [INFO] --- maven-install-plugin:2.3.1:install (default-install) @ checkstyle --- [INFO] Installing /opt/cloudstack/cloudstack/tools/checkstyle/target/checkstyle-4.4.0-SNAPSHOT.jar to /root/.m2/repository/org/apache/cloudstack/checkstyle/4.4.0-SNAPSHOT/checkstyle-4.4.0-SNAPSHOT.jar [INFO] Installing /opt/cloudstack/cloudstack/tools/checkstyle/pom.xml to /root/.m2/repository/org/apache/cloudstack/checkstyle/4.4.0-SNAPSHOT/checkstyle-4.4.0-SNAPSHOT.pom [INFO] [INFO] [INFO] Building Apache CloudStack 4.4.0-SNAPSHOT [INFO] [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloudstack --- [INFO] Deleting /opt/cloudstack/cloudstack/target (includes = [**/*], excludes = []) [INFO] Deleting /opt/cloudstack/cloudstack (includes = [target, dist], excludes = []) [INFO] [INFO] --- maven-remote-resources-plugin:1.3:process (default) @ cloudstack --- [INFO] [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ cloudstack --- [INFO] [INFO] --- maven-install-plugin:2.3.1:install (default-install) @ cloudstack --- [INFO] Installing /opt/cloudstack/cloudstack/pom.xml to /root/.m2/repository/org/apache/cloudstack/cloudstack/4.4.0-SNAPSHOT/cloudstack-4.4.0-SNAPSHOT.pom [INFO] [INFO] [INFO] Building Apache CloudStack Maven Conventions Parent 4.4.0-SNAPSHOT [INFO] [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-maven-standard --- [INFO] Deleting /opt/cloudstack/cloudstack/maven-standard/target (includes = [**/*], excludes = []) [INFO] Deleting /opt/cloudstack/cloudstack/maven-standard (includes = [target, dist], excludes = []) [INFO] [INFO] --- maven-checkstyle-plugin:2.11:check (cloudstack-checkstyle) @ cloud-maven-standard --- [INFO] Starting audit... Audit done. [INFO] [INFO] ---
RE: Code quality, QA, etc
+1 for pre- commit testing. Whichever tool enforces it would be good choice. For feature check in, we ( community) require sanity tests to be submitted by feature owners and this was followed well in 4.0 release but there is lapse in this practice now. This would be a great if RM can enforce this during check ins - review unit tests and results before approving a check in. -Original Message- From: Trippie [mailto:trip...@gmail.com] On Behalf Of Hugo Trippaers Sent: Friday, February 07, 2014 12:46 AM To: dev Cc: jfarr...@apache.org Subject: Re: Code quality, QA, etc Hey David, I would make a distinction between code issues and functional issues. Occasionally somebody just plainly breaks the build, i'm guilty of that myself actually, and thats just plain stupid. Luckily we have Jenkins to catch these errors quickly. I'm in a continuous struggle with Jenkins to get the build time to less than 5 minutes. I feel that is an acceptable time to get feedback on a commit, any longer and you have moved on to the next thing or gone home. Also this kind of testing isn't really hard, run the build and unit tests. By introducing something like gerrit we can actually make this happen before committing it to the repo. Push a patch to gerrit, gerrit tells jenkins to test the patch, if +1 from jerkins commit, for non committers the step would be to invite somebody for review as well. Second nice thing about jenkins is the post-review test, if a contributor submits a patch its build by jenkins, if a reviewes approves the patch, jerkins will again run a build to ensure that the patch will still apply and doesn't break the build. Very handy if there is some time between patch submission and patch review. Functional issues are much harder to track. For example yesterday i found several issues in the contrail plugin that would not cause any pain in a contrail environment, but any other environments creating a network would fail. These examples are too common and difficult to catch with unit tests. It can be done, but requires some serious effort on the developers side and we in general don't seem to be very active at writing unit tests. These kind of issues can only be found by actually running CloudStack and executing a series of functional tests. Ideally that is what we have the BVT suite for, but i think our current BVT setup is not documented enough to give accurate feedback to a developer about which patch broke a certain piece of functionality. In jenkins the path from code to BVT is not kept yet, so it is almost impossible to see which commits were new in a particular run of the bvt suite. Personally i'm trying to get into the habit of running a series of tests on devcloud before committing something. Doesn't prove a lot, but does guarantee that the bare basic developer functionality is working before committing something. After a commit at least i'm sure that anybody will be able to spinup devcloud and deploy an instance. I'm trying to get this automated as well so we can use this as feedback on a patch. Beers for anyone who writes an easy to use script that configures devcloud with a zone and tests if a user vm can be instantiated on an isolated sourcenat network. If we could include such a script in the tree it might help people with testing their patch before committing. I think we are seeing more and more reverts in the tree. Not necessarily a good thing, but at least people know that there is that option if a commit really breaks a build. Also please help each other out, everybody can make a mistake and commit it. If its a trivial mistake it might not be much effort to track it down and fix it, which is way better than a revert or a mail that something is broken. In short, we need to make testing more efficient and transparent to allow people to easily incorporate it in their personal workflow. Cheers, Hugo On 7 feb. 2014, at 04:50, David Nalley da...@gnsa.us wrote: Hi folks, We continue to break things large and small in the codebase, and after a number of different conversations; I thought I'd bring that discussion here. First - coding quality is only one factor that the PMC considers when making someone a committer. Second - CloudStack is a huge codebase; has a ton of inter-related pieces, and unintended consequences are easy. We also have an pretty heady commit velocity - 20+ commits today alone. Some communities have Review-then-commit - which would slow us down, and presumably help us increase quality. However, I am not personally convinced that it will do so measurably because even the most experienced CloudStack developers occasionally break a build or worse. We could have an automated pipeline that verifies a number of different tests pass - before a patch/commit makes it into a mainline branch. That is difficult with our current tooling; but perhaps something worth considering. At FOSDEM, Hugo
RE: Source build failing in DevCloud
What is the output of java -version . Try after installing Java c 1.7 .It might resolve this issue. Regards Sadhu -Original Message- From: CK [mailto:cloudw...@gmail.com] Sent: 07 February 2014 17:11 To: dev@cloudstack.apache.org Subject: Source build failing in DevCloud Hi, This is my first attempt at building CloudStack from the source code and decided on using DevCloud. I followed the wiki for using DevCloud https://cwiki.apache.org/confluence/display/CLOUDSTACK/DevCloud. I want to build the 4.3 release and used the updated DevCloud 2.0 appliance and as per the instructions. I git cloned the latest master code, but when I then run mvn -P developer,systemvm clean install. I get a compilation/build failure (see output below). Q1) Have I missed something and why I am I getting this error? Q2) I see it is building 4.4.0-SNAPSHOT how do I get it to build 4.3? Thanks. [INFO] [INFO] Building Apache CloudStack Developer Tools - Checkstyle Configuration 4.4.0-SNAPSHOT [INFO] [INFO] [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ checkstyle --- [INFO] Deleting /opt/cloudstack/cloudstack/tools/checkstyle/target [INFO] [INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ checkstyle --- [debug] execute contextualize [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] Copying 1 resource [INFO] [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ checkstyle --- [INFO] No sources to compile [INFO] [INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ checkstyle --- [debug] execute contextualize [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory /opt/cloudstack/cloudstack/tools/checkstyle/src/test/resources [INFO] [INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ checkstyle --- [INFO] No sources to compile [INFO] [INFO] --- maven-surefire-plugin:2.10:test (default-test) @ checkstyle --- [INFO] No tests to run. [INFO] Surefire report directory: /opt/cloudstack/cloudstack/tools/checkstyle/target/surefire-reports --- T E S T S --- Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] --- maven-jar-plugin:2.3.2:jar (default-jar) @ checkstyle --- [INFO] Building jar: /opt/cloudstack/cloudstack/tools/checkstyle/target/checkstyle-4.4.0-SNAPSHOT.jar [INFO] [INFO] --- maven-install-plugin:2.3.1:install (default-install) @ checkstyle --- [INFO] Installing /opt/cloudstack/cloudstack/tools/checkstyle/target/checkstyle-4.4.0-SNAPSHOT.jar to /root/.m2/repository/org/apache/cloudstack/checkstyle/4.4.0-SNAPSHOT/checkstyle-4.4.0-SNAPSHOT.jar [INFO] Installing /opt/cloudstack/cloudstack/tools/checkstyle/pom.xml to /root/.m2/repository/org/apache/cloudstack/checkstyle/4.4.0-SNAPSHOT/checkstyle-4.4.0-SNAPSHOT.pom [INFO] [INFO] [INFO] Building Apache CloudStack 4.4.0-SNAPSHOT [INFO] [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloudstack --- [INFO] Deleting /opt/cloudstack/cloudstack/target (includes = [**/*], excludes = []) [INFO] Deleting /opt/cloudstack/cloudstack (includes = [target, dist], excludes = []) [INFO] [INFO] --- maven-remote-resources-plugin:1.3:process (default) @ cloudstack --- [INFO] [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ cloudstack --- [INFO] [INFO] --- maven-install-plugin:2.3.1:install (default-install) @ cloudstack --- [INFO] Installing /opt/cloudstack/cloudstack/pom.xml to /root/.m2/repository/org/apache/cloudstack/cloudstack/4.4.0-SNAPSHOT/cloudstack-4.4.0-SNAPSHOT.pom [INFO] [INFO] [INFO] Building Apache CloudStack Maven Conventions Parent 4.4.0-SNAPSHOT [INFO] [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-maven-standard --- [INFO] Deleting /opt/cloudstack/cloudstack/maven-standard/target (includes = [**/*], excludes = []) [INFO] Deleting /opt/cloudstack/cloudstack/maven-standard (includes = [target, dist], excludes = []) [INFO] [INFO] --- maven-checkstyle-plugin:2.11:check (cloudstack-checkstyle) @ cloud-maven-standard --- [INFO] Starting audit... Audit done. [INFO] [INFO] --- maven-remote-resources-plugin:1.3:process (default) @ cloud-maven-standard --- [INFO] [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ cloud-maven-standard ---
Re: Not able to add primary Storage Cloudstack 4.3
Hi Rajesh, Thanks for providing API. I have successfully attached primary storage to cloudstack. I am facing problem with SSVM and CPVM also, probably I will initiate new mail-trail ssytem vm issues. Reards, Tejas On Thu, Feb 6, 2014 at 12:27 PM, Rajesh Battala rajesh.batt...@citrix.comwrote: Tejas, This is how the api command looks like to create primary and secondary storage. command=createStoragePoolzoneid=581bc2b6-0295-40e0-90ed-2717e239c2b6podId=896cff2e-d51e-454b-97ed-cab078bf3be2clusterid=3785c58a-d891-4482-9e6f-72f2060b80d3name=psscope=clusterurl=cifs%3A%2F%2F10.102.192.150%2Fsmb-share%2Frajesh%2Fprimary%3Fuser%3DAdministrator%26domain%3DBLRresponse=jsonsessionkey=1L5raRYhxHCAgXHJ1WGvLjs7Zws%3D_=1391666896777 command=addImageStoreresponse=jsonsessionkey=1L5raRYhxHCAgXHJ1WGvLjs7Zws%3Dname=secprovider=SMBzoneid=581bc2b6-0295-40e0-90ed-2717e239c2b6url=cifs%3A%2F%2F10.102.192.150%2Fsmb-share%2Frajesh%2Fsecondary%3Fuser%3DAdministrator%26domain%3DBLR_=1391666908684 in the wizard specify the path as /smb-share/rajesh/primary Similar for secondary. Provide credentials and domain name in the wizard. Make sure you were able to access the share from the hyperv hosts before adding them to CS. Thanks Rajesh Battala *From:* Tejas Gadaria [mailto:refond.g...@gmail.com] *Sent:* Thursday, February 6, 2014 11:03 AM *To:* us...@cloudstack.apache.org *Cc:* Rajesh Battala; dev@cloudstack.apache.org *Subject:* Re: Not able to add primary Storage Cloudstack 4.3 Hi, I am not able to attach primary storage in Cloudstack as cifs. So what I am doing wrong OR what are the issues with primary storage.? Regards Tejas On Wed, Feb 5, 2014 at 4:35 PM, Tejas Gadaria refond.g...@gmail.com wrote: Hi Paul, forgot to mention, hyper-v agent is running. Regards, Tejas On Wed, Feb 5, 2014 at 4:33 PM, Tejas Gadaria refond.g...@gmail.com wrote: Hi Paul, yesterday when I was trying to add secondary storage with /hcloud/secondary , In logs I was not able to see much activity, but storage got attached. I have uploaded screenshot in previous mail. I have reattached the secondary storage, and it logs are Trying to add a new data store at cifs:// 10.129.151.55/hcloud/secondary?user=administratordomain=nwx.com to data center 3 2014-02-05 15:36:07,825 DEBUG [c.c.u.UriUtils] (catalina-exec-20:ctx-a19af0fa ctx-fb83ed9b) foundUser istrue 2014-02-05 15:36:07,825 DEBUG [c.c.u.UriUtils] (catalina-exec-20:ctx-a19af0fa ctx-fb83ed9b) foundPswd istrue I have not enable the zone yet. In attached screenshot of secondary storage in previous mail protocol:cifs provider : NFS I suspect, it because I have taken chunk of NFS datastore, mounted and share it via Windows fie server (Standalone) please find logs in attached file. Regards, Tejas On Tue, Feb 4, 2014 at 5:52 PM, Paul Angus paul.an...@shapeblue.com wrote: Hey Tejas, cifs%3A%2F%2F10.129.151.155hcloud%2Fsecondary%3Fuser%3Dadministrator%26domain%3Dnwx.com Theres a / (%2F) missing between 10.129.151.155 and hcloud - my recollection is that primary and secondary storage UIs are not consistent, add the secondary path as /hcloud/secondary re primary - is the agent service running? it doesn't seem to (always) automatically start even when it's set to automatic in services. Regards, Paul Angus Cloud Architect S: +44 20 3603 0540 | M: +447711418784 | T: @CloudyAngus paul.an...@shapeblue.com From: Tejas Gadaria [mailto:refond.g...@gmail.com] Sent: 04 February 2014 12:15 To: us...@cloudstack.apache.org Cc: rajesh.batt...@citrix.com; dev@cloudstack.apache.org Subject: Re: Not able to add primary Storage Cloudstack 4.3 Hi Paul, I tried with hcloud/primary and primary storage shows Unsupported command issued. secondary storage shows Failed to add data store: cifs URI missing host and/or path. Make sure it's of the format cifs://hostname/path?user=usernamepassword=password After that I was trying to add secondary storage with /hcloud/secondary, I didn't show much activity in log. In Cloudstack console, secondary storage got added, but protocol is CIFS and provider is NFS. Please find attached logs. and screenshot from attached secondary storage. Regards, Tejas On Tue, Feb 4, 2014 at 12:29 PM, Paul Angus paul.an...@shapeblue.com mailto:paul.an...@shapeblue.com wrote: Tejas, You have hcloud\primary, it should be hcloud/primary Regards Paul Angus Cloud Architect S: +44 20 3603 0540tel:%2B44%2020%203603%200540%2B44%2020%203603%200540 | M: +447711418784tel:%2B447711418784 %2B447711418784 | T: CloudyAngus paul.an...@shapeblue.commailto:paul.an...@shapeblue.commailto: paul.an...@shapeblue.commailto:paul.an...@shapeblue.com From: Tejas Gadaria [mailto:refond.g...@gmail.commailto: refond.g...@gmail.com] Sent: 04 February 2014 05:16 To: us...@cloudstack.apache.orgmailto:us...@cloudstack.apache.org Cc:
Re: Source build failing in DevCloud
Java version is: javac 1.6.0_24 I'm new to 'git', how do I ...checkout the 4.3 branch, git pull then compile you should get the 4.3 current build., can you provide the commands? Thanks On 7 February 2014 12:00, Alex Hitchins alex.hitch...@shapeblue.com wrote: I've had a quick look at the error statements - what version of Java is on the devcloud vm? Also, if you checkout the 4.3 branch, git pull then compile you should get the 4.3 current build. Regards Alex Hitchins D: +44 1892 523 587 | S: +44 20 3603 0540 | M: +44 7788 423 969 alex.hitch...@shapeblue.com -Original Message- From: CK [mailto:cloudw...@gmail.com] Sent: 07 February 2014 11:41 To: dev@cloudstack.apache.org Subject: Source build failing in DevCloud Hi, This is my first attempt at building CloudStack from the source code and decided on using DevCloud. I followed the wiki for using DevCloud https://cwiki.apache.org/confluence/display/CLOUDSTACK/DevCloud. I want to build the 4.3 release and used the updated DevCloud 2.0 appliance and as per the instructions. I git cloned the latest master code, but when I then run mvn -P developer,systemvm clean install. I get a compilation/build failure (see output below). Q1) Have I missed something and why I am I getting this error? Q2) I see it is building 4.4.0-SNAPSHOT how do I get it to build 4.3? Thanks. [INFO] [INFO] Building Apache CloudStack Developer Tools - Checkstyle Configuration 4.4.0-SNAPSHOT [INFO] [INFO] [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ checkstyle --- [INFO] Deleting /opt/cloudstack/cloudstack/tools/checkstyle/target [INFO] [INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ checkstyle --- [debug] execute contextualize [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] Copying 1 resource [INFO] [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ checkstyle --- [INFO] No sources to compile [INFO] [INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ checkstyle --- [debug] execute contextualize [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory /opt/cloudstack/cloudstack/tools/checkstyle/src/test/resources [INFO] [INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ checkstyle --- [INFO] No sources to compile [INFO] [INFO] --- maven-surefire-plugin:2.10:test (default-test) @ checkstyle --- [INFO] No tests to run. [INFO] Surefire report directory: /opt/cloudstack/cloudstack/tools/checkstyle/target/surefire-reports --- T E S T S --- Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] --- maven-jar-plugin:2.3.2:jar (default-jar) @ checkstyle --- [INFO] Building jar: /opt/cloudstack/cloudstack/tools/checkstyle/target/checkstyle-4.4.0-SNAPSHOT.jar [INFO] [INFO] --- maven-install-plugin:2.3.1:install (default-install) @ checkstyle --- [INFO] Installing /opt/cloudstack/cloudstack/tools/checkstyle/target/checkstyle-4.4.0-SNAPSHOT.jar to /root/.m2/repository/org/apache/cloudstack/checkstyle/4.4.0-SNAPSHOT/checkstyle-4.4.0-SNAPSHOT.jar [INFO] Installing /opt/cloudstack/cloudstack/tools/checkstyle/pom.xml to /root/.m2/repository/org/apache/cloudstack/checkstyle/4.4.0-SNAPSHOT/checkstyle-4.4.0-SNAPSHOT.pom [INFO] [INFO] [INFO] Building Apache CloudStack 4.4.0-SNAPSHOT [INFO] [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloudstack --- [INFO] Deleting /opt/cloudstack/cloudstack/target (includes = [**/*], excludes = []) [INFO] Deleting /opt/cloudstack/cloudstack (includes = [target, dist], excludes = []) [INFO] [INFO] --- maven-remote-resources-plugin:1.3:process (default) @ cloudstack --- [INFO] [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ cloudstack --- [INFO] [INFO] --- maven-install-plugin:2.3.1:install (default-install) @ cloudstack --- [INFO] Installing /opt/cloudstack/cloudstack/pom.xml to /root/.m2/repository/org/apache/cloudstack/cloudstack/4.4.0-SNAPSHOT/cloudstack-4.4.0-SNAPSHOT.pom [INFO] [INFO] [INFO] Building Apache CloudStack Maven Conventions Parent 4.4.0-SNAPSHOT [INFO] [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-maven-standard --- [INFO] Deleting
RE: systemvm-4.3.0.iso
Hi Paul,
Looks like you are using untagged vlan in your setup. There was a bug in agent
code with respect to that. Rajesh made a fix for it in commit-id
7df148c762510f669b60edff9d8d690c800b1f95 few days back. This fix should address
the problem.
Regards,
Devdeep
-Original Message-
From: Paul Angus [mailto:paul.an...@shapeblue.com]
Sent: Friday, February 7, 2014 4:55 PM
To: dev@cloudstack.apache.org
Subject: RE: systemvm-4.3.0.iso
Thanks for the info Devdeep,
so yes the systemvm iso is getting copied to the local disk, but I'm getting
the following error:
ERROR HypervResource.HypervResourceController
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - com.cloud.agent.api.StartCommand fail
on exceptionObject reference not set to an instance of an object.
System.NullReferenceException: Object reference not set to an instance of an
object.
at HypervResource.WmiCallsV2.DeployVirtualMachine(Object jsonObj, String
systemVmIso)
at HypervResource.HypervResourceController.StartCommand(Object cmd)
agent log:
2014-02-07 11:19:56,685 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Going ahead with create VM s-27-VM, 1
vcpus, 512MB RAM
2014-02-07 11:19:56,716 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Created VM s-27-VM
2014-02-07 11:19:57,075 [28] INFO HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - VM with display name s-27-VM has GUID
9E6D3606-8210-4E84-8796-BEB7970785F4
2014-02-07 11:19:57,075 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Resources for vm s-27-VM: 512 MB
memory, 1 vcpus
2014-02-07 11:19:57,106 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - New controller type
Microsoft:Hyper-V:Synthetic SCSI Controller WMI path is
\\WIN-G23HSIAIU4O\root\virtualization\v2:Msvm_ResourceAllocationSettingData.InstanceID=Microsoft:9E6D3606-8210-4E84-8796-BEB7970785F4\\57035109-ED28-4F65-8BCB-487CFC66F8E8\\0s
2014-02-07 11:19:57,106 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Going to create s-27-VM with attached
voluem ROOT-27 at \\10.0.1.27\hypervPri\stor\Primary\ROOT-27.vhd
2014-02-07 11:19:57,106 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Create disk type
Microsoft:Hyper-V:Synthetic Disk Drive (Named: ROOT-27), on vm s-27-VM ,
inserting disk\\10.0.1.27\hypervPri\stor\Primary\ROOT-27.vhd
2014-02-07 11:19:57,106 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Creating DISK for VM s-27-VM (GUID
9E6D3606-8210-4E84-8796-BEB7970785F4) by attaching
\\10.0.1.27\hypervPri\stor\Primary\ROOT-27.vhd
2014-02-07 11:19:57,403 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Creating disk drive type
Microsoft:Hyper-V:Synthetic Disk Drive, parent IDE controller is
\\WIN-G23HSIAIU4O\root\virtualization\v2:Msvm_ResourceAllocationSettingData.InstanceID=Microsoft:9E6D3606-8210-4E84-8796-BEB7970785F4\\83F8638B-8DCA-4152-9EDA-2CA8B33039B4\\0
and address on controller is 0
2014-02-07 11:19:57,418 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - New disk drive type
Microsoft:Hyper-V:Synthetic Disk Drive WMI path is
\\WIN-G23HSIAIU4O\root\virtualization\v2:Msvm_ResourceAllocationSettingData.InstanceID=Microsoft:9E6D3606-8210-4E84-8796-BEB7970785F4\\83F8638B-8DCA-4152-9EDA-2CA8B33039B4\\0\\0\\Ds
2014-02-07 11:19:57,574 [28] INFO HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Created disk
\\WIN-G23HSIAIU4O\root\virtualization\v2:Msvm_StorageAllocationSettingData.InstanceID=Microsoft:9E6D3606-8210-4E84-8796-BEB7970785F4\\83F8638B-8DCA-4152-9EDA-2CA8B33039B4\\0\\0\\L
for VM s-27-VM (GUID 9E6D3606-8210-4E84-8796-BEB7970785F4), image
\\10.0.1.27\hypervPri\stor\Primary\ROOT-27.vhd
2014-02-07 11:19:57,574 [28] ERROR HypervResource.HypervResourceController
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - com.cloud.agent.api.StartCommand fail
on exceptionObject reference not set to an instance of an object.
System.NullReferenceException: Object reference not set to an instance of an
object.
at HypervResource.WmiCallsV2.DeployVirtualMachine(Object jsonObj, String
systemVmIso)
at HypervResource.HypervResourceController.StartCommand(Object cmd)
2014-02-07 11:19:57,574 [28] INFO HypervResource.HypervResourceController
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - {
com.cloud.agent.api.StartAnswer: {
result: false,
details: com.cloud.agent.api.StartCommand fail on exceptionObject
reference not set to an instance of an object.,
vm: {
id: 27,
name: s-27-VM,
type: SecondaryStorageVm,
cpus: 1,
minSpeed: 125,
maxSpeed: 500,
minRam: 536870912,
maxRam: 536870912,
arch: x86_64,
os: Debian GNU/Linux 5.0 (64-bit),
bootArgs: template=domP type=secstorage host=10.0.100.4 port=8250
name=s-27-VM zone=1 pod=1 guid=s-27-VM
RE: Source build failing in DevCloud
I haven't used devcloud in a while - this link might prove useful to you. http://cloudstack.apache.org/developers.html After the clone is complete, you can git checkout 4.3 then git pull to ensure it's updated. The link gives you the steps to get your first commit done. Welcome to the group! Regards Alex Hitchins D: +44 1892 523 587 | S: +44 20 3603 0540 | M: +44 7788 423 969 alex.hitch...@shapeblue.com -Original Message- From: CK [mailto:cloudw...@gmail.com] Sent: 07 February 2014 12:05 To: dev@cloudstack.apache.org Subject: Re: Source build failing in DevCloud Java version is: javac 1.6.0_24 I'm new to 'git', how do I ...checkout the 4.3 branch, git pull then compile you should get the 4.3 current build., can you provide the commands? Thanks On 7 February 2014 12:00, Alex Hitchins alex.hitch...@shapeblue.com wrote: I've had a quick look at the error statements - what version of Java is on the devcloud vm? Also, if you checkout the 4.3 branch, git pull then compile you should get the 4.3 current build. Regards Alex Hitchins D: +44 1892 523 587 | S: +44 20 3603 0540 | M: +44 7788 423 969 alex.hitch...@shapeblue.com -Original Message- From: CK [mailto:cloudw...@gmail.com] Sent: 07 February 2014 11:41 To: dev@cloudstack.apache.org Subject: Source build failing in DevCloud Hi, This is my first attempt at building CloudStack from the source code and decided on using DevCloud. I followed the wiki for using DevCloud https://cwiki.apache.org/confluence/display/CLOUDSTACK/DevCloud. I want to build the 4.3 release and used the updated DevCloud 2.0 appliance and as per the instructions. I git cloned the latest master code, but when I then run mvn -P developer,systemvm clean install. I get a compilation/build failure (see output below). Q1) Have I missed something and why I am I getting this error? Q2) I see it is building 4.4.0-SNAPSHOT how do I get it to build 4.3? Thanks. [INFO] -- -- [INFO] Building Apache CloudStack Developer Tools - Checkstyle Configuration 4.4.0-SNAPSHOT [INFO] -- -- [INFO] [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ checkstyle --- [INFO] Deleting /opt/cloudstack/cloudstack/tools/checkstyle/target [INFO] [INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ checkstyle --- [debug] execute contextualize [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] Copying 1 resource [INFO] [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ checkstyle --- [INFO] No sources to compile [INFO] [INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ checkstyle --- [debug] execute contextualize [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory /opt/cloudstack/cloudstack/tools/checkstyle/src/test/resources [INFO] [INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ checkstyle --- [INFO] No sources to compile [INFO] [INFO] --- maven-surefire-plugin:2.10:test (default-test) @ checkstyle --- [INFO] No tests to run. [INFO] Surefire report directory: /opt/cloudstack/cloudstack/tools/checkstyle/target/surefire-reports --- T E S T S --- Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] --- maven-jar-plugin:2.3.2:jar (default-jar) @ checkstyle --- [INFO] Building jar: /opt/cloudstack/cloudstack/tools/checkstyle/target/checkstyle-4.4.0-SN APSHOT.jar [INFO] [INFO] --- maven-install-plugin:2.3.1:install (default-install) @ checkstyle --- [INFO] Installing /opt/cloudstack/cloudstack/tools/checkstyle/target/checkstyle-4.4.0-SN APSHOT.jar to /root/.m2/repository/org/apache/cloudstack/checkstyle/4.4.0-SNAPSHOT/c heckstyle-4.4.0-SNAPSHOT.jar [INFO] Installing /opt/cloudstack/cloudstack/tools/checkstyle/pom.xml to /root/.m2/repository/org/apache/cloudstack/checkstyle/4.4.0-SNAPSHOT/c heckstyle-4.4.0-SNAPSHOT.pom [INFO] [INFO] -- -- [INFO] Building Apache CloudStack 4.4.0-SNAPSHOT [INFO] -- -- [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloudstack --- [INFO] Deleting /opt/cloudstack/cloudstack/target (includes = [**/*], excludes = []) [INFO] Deleting /opt/cloudstack/cloudstack (includes = [target, dist], excludes = []) [INFO] [INFO] --- maven-remote-resources-plugin:1.3:process (default) @ cloudstack --- [INFO] [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ cloudstack --- [INFO] [INFO]
RE: Source build failing in DevCloud
Also, I think that Java version might be old - where did you get the devcloud image from? Regards Alex Hitchins D: +44 1892 523 587 | S: +44 20 3603 0540 | M: +44 7788 423 969 alex.hitch...@shapeblue.com -Original Message- From: CK [mailto:cloudw...@gmail.com] Sent: 07 February 2014 12:05 To: dev@cloudstack.apache.org Subject: Re: Source build failing in DevCloud Java version is: javac 1.6.0_24 I'm new to 'git', how do I ...checkout the 4.3 branch, git pull then compile you should get the 4.3 current build., can you provide the commands? Thanks On 7 February 2014 12:00, Alex Hitchins alex.hitch...@shapeblue.com wrote: I've had a quick look at the error statements - what version of Java is on the devcloud vm? Also, if you checkout the 4.3 branch, git pull then compile you should get the 4.3 current build. Regards Alex Hitchins D: +44 1892 523 587 | S: +44 20 3603 0540 | M: +44 7788 423 969 alex.hitch...@shapeblue.com -Original Message- From: CK [mailto:cloudw...@gmail.com] Sent: 07 February 2014 11:41 To: dev@cloudstack.apache.org Subject: Source build failing in DevCloud Hi, This is my first attempt at building CloudStack from the source code and decided on using DevCloud. I followed the wiki for using DevCloud https://cwiki.apache.org/confluence/display/CLOUDSTACK/DevCloud. I want to build the 4.3 release and used the updated DevCloud 2.0 appliance and as per the instructions. I git cloned the latest master code, but when I then run mvn -P developer,systemvm clean install. I get a compilation/build failure (see output below). Q1) Have I missed something and why I am I getting this error? Q2) I see it is building 4.4.0-SNAPSHOT how do I get it to build 4.3? Thanks. [INFO] -- -- [INFO] Building Apache CloudStack Developer Tools - Checkstyle Configuration 4.4.0-SNAPSHOT [INFO] -- -- [INFO] [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ checkstyle --- [INFO] Deleting /opt/cloudstack/cloudstack/tools/checkstyle/target [INFO] [INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ checkstyle --- [debug] execute contextualize [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] Copying 1 resource [INFO] [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ checkstyle --- [INFO] No sources to compile [INFO] [INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ checkstyle --- [debug] execute contextualize [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory /opt/cloudstack/cloudstack/tools/checkstyle/src/test/resources [INFO] [INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ checkstyle --- [INFO] No sources to compile [INFO] [INFO] --- maven-surefire-plugin:2.10:test (default-test) @ checkstyle --- [INFO] No tests to run. [INFO] Surefire report directory: /opt/cloudstack/cloudstack/tools/checkstyle/target/surefire-reports --- T E S T S --- Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] --- maven-jar-plugin:2.3.2:jar (default-jar) @ checkstyle --- [INFO] Building jar: /opt/cloudstack/cloudstack/tools/checkstyle/target/checkstyle-4.4.0-SN APSHOT.jar [INFO] [INFO] --- maven-install-plugin:2.3.1:install (default-install) @ checkstyle --- [INFO] Installing /opt/cloudstack/cloudstack/tools/checkstyle/target/checkstyle-4.4.0-SN APSHOT.jar to /root/.m2/repository/org/apache/cloudstack/checkstyle/4.4.0-SNAPSHOT/c heckstyle-4.4.0-SNAPSHOT.jar [INFO] Installing /opt/cloudstack/cloudstack/tools/checkstyle/pom.xml to /root/.m2/repository/org/apache/cloudstack/checkstyle/4.4.0-SNAPSHOT/c heckstyle-4.4.0-SNAPSHOT.pom [INFO] [INFO] -- -- [INFO] Building Apache CloudStack 4.4.0-SNAPSHOT [INFO] -- -- [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloudstack --- [INFO] Deleting /opt/cloudstack/cloudstack/target (includes = [**/*], excludes = []) [INFO] Deleting /opt/cloudstack/cloudstack (includes = [target, dist], excludes = []) [INFO] [INFO] --- maven-remote-resources-plugin:1.3:process (default) @ cloudstack --- [INFO] [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ cloudstack --- [INFO] [INFO] --- maven-install-plugin:2.3.1:install (default-install) @ cloudstack --- [INFO] Installing /opt/cloudstack/cloudstack/pom.xml to /root/.m2/repository/org/apache/cloudstack/cloudstack/4.4.0-SNAPSHOT/c
[DISCUSS]atomic unit tests
LS,
I had a discussion on uniq paths and unit tests yesterday night with
our new colleague Miguel about the unit tests that we have in
Cloudstack at the moment. He pointed me to a common smell that I
overlooked so far in Cloudstack.
A lot of times unit tests are organised as folows:
class test{
@Before
someInitThingies(){}
someTest(){}
someOtherTest(){}
@Test
doTheStuff(){
someTest();
someOtherTest();
}
}// end of unit test class
this (anti) pattern defeats the purpose of unit testing a bit. You
will not get report on the individual use cases of your methods. I
would call upon all to implement the class as folows:
class test{
@Before
someInitThingies(){}
@Test
someTest(){}
@Test
someOtherTest(){}
}// end of unit test class
ASAIROCS there is no pattern or use that justifies putting the test
methods in a common placeholder. This is up to the test framework and
it needs it to give usefull debugging reports.
any thoughts/comment welcome
--
Daan
Review Request 17843: CLOUDSTACK-6058: Update the java binding used to the latest one that came with XenServer 6.2 sp1
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17843/ --- Review request for cloudstack, Alex Huang, Devdeep Singh, and Koushik Das. Bugs: CLOUDSTACK-6058 https://issues.apache.org/jira/browse/CLOUDSTACK-6058 Repository: cloudstack-git Description --- Update the java bindings that are used in cloudstack with the latest one. There are two parts for this change, 1st one is to replace the java bindings; and second part is to add the customization's made in xenapi for Cloudstack. 1. https://reviews.apache.org/r/17843/ - Replaces the java bindings. 2. https://reviews.apache.org/r/17844/ - Customizations in xenapi for CloudStack. Diffs - deps/XenServerJava/src/LICENSE.Apache-2.0.txt PRE-CREATION deps/XenServerJava/src/LICENSE.txt PRE-CREATION deps/XenServerJava/src/Makefile PRE-CREATION deps/XenServerJava/src/README.txt PRE-CREATION deps/XenServerJava/src/com/xensource/xenapi/APIVersion.java 8d7247d deps/XenServerJava/src/com/xensource/xenapi/Auth.java cffaaac deps/XenServerJava/src/com/xensource/xenapi/Blob.java 494b928 deps/XenServerJava/src/com/xensource/xenapi/Bond.java 935fd98 deps/XenServerJava/src/com/xensource/xenapi/Connection.java a652450 deps/XenServerJava/src/com/xensource/xenapi/Console.java ac2c9a7 deps/XenServerJava/src/com/xensource/xenapi/Crashdump.java be11588 deps/XenServerJava/src/com/xensource/xenapi/DRTask.java 14a5e40 deps/XenServerJava/src/com/xensource/xenapi/DataSource.java 5f777a3 deps/XenServerJava/src/com/xensource/xenapi/Event.java ec43814 deps/XenServerJava/src/com/xensource/xenapi/GPUGroup.java 9070446 deps/XenServerJava/src/com/xensource/xenapi/Host.java 2195048 deps/XenServerJava/src/com/xensource/xenapi/HostCpu.java 15fcfa7 deps/XenServerJava/src/com/xensource/xenapi/HostCrashdump.java 02c3bd9 deps/XenServerJava/src/com/xensource/xenapi/HostMetrics.java b41277a deps/XenServerJava/src/com/xensource/xenapi/HostPatch.java fa77588 deps/XenServerJava/src/com/xensource/xenapi/Marshalling.java 865bc16 deps/XenServerJava/src/com/xensource/xenapi/Message.java a3658e3 deps/XenServerJava/src/com/xensource/xenapi/Network.java e9194ef deps/XenServerJava/src/com/xensource/xenapi/PBD.java d289824 deps/XenServerJava/src/com/xensource/xenapi/PCI.java e11230a deps/XenServerJava/src/com/xensource/xenapi/PGPU.java 0f5ddb4 deps/XenServerJava/src/com/xensource/xenapi/PIF.java 27618b4 deps/XenServerJava/src/com/xensource/xenapi/PIFMetrics.java ab6268e deps/XenServerJava/src/com/xensource/xenapi/Pool.java 7ceebb0 deps/XenServerJava/src/com/xensource/xenapi/PoolPatch.java 7361bc7 deps/XenServerJava/src/com/xensource/xenapi/Role.java 5808b13 deps/XenServerJava/src/com/xensource/xenapi/SM.java 0ec3626 deps/XenServerJava/src/com/xensource/xenapi/SR.java db6770f deps/XenServerJava/src/com/xensource/xenapi/Secret.java 870725f deps/XenServerJava/src/com/xensource/xenapi/Session.java bdf3b4c deps/XenServerJava/src/com/xensource/xenapi/Subject.java a599f237 deps/XenServerJava/src/com/xensource/xenapi/Task.java 59d59f6 deps/XenServerJava/src/com/xensource/xenapi/Tunnel.java 44b236d deps/XenServerJava/src/com/xensource/xenapi/Types.java 777d580 deps/XenServerJava/src/com/xensource/xenapi/User.java dff27f5 deps/XenServerJava/src/com/xensource/xenapi/VBD.java 56e6f4b deps/XenServerJava/src/com/xensource/xenapi/VBDMetrics.java 1f678fa deps/XenServerJava/src/com/xensource/xenapi/VDI.java 992a6a6 deps/XenServerJava/src/com/xensource/xenapi/VGPU.java 19c88fd deps/XenServerJava/src/com/xensource/xenapi/VGPUType.java PRE-CREATION deps/XenServerJava/src/com/xensource/xenapi/VIF.java 8302800 deps/XenServerJava/src/com/xensource/xenapi/VIFMetrics.java 0c17217 deps/XenServerJava/src/com/xensource/xenapi/VLAN.java 9bea039 deps/XenServerJava/src/com/xensource/xenapi/VM.java cfa5e6c deps/XenServerJava/src/com/xensource/xenapi/VMAppliance.java b7e7e36 deps/XenServerJava/src/com/xensource/xenapi/VMGuestMetrics.java 08d88e3 deps/XenServerJava/src/com/xensource/xenapi/VMMetrics.java 5f9a7a4 deps/XenServerJava/src/com/xensource/xenapi/VMPP.java 0ee0b3e deps/XenServerJava/src/com/xensource/xenapi/VTPM.java 25c3320 deps/XenServerJava/src/com/xensource/xenapi/XenAPIObject.java b9821ee Diff: https://reviews.apache.org/r/17843/diff/ Testing --- Thanks, Sanjay Tripathi
Review Request 17844: CLOUDSTACK-6058: Update the java binding used to the latest one that came with XenServer 6.2 sp1.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17844/ --- Review request for cloudstack, Alex Huang, Devdeep Singh, and Koushik Das. Bugs: CLOUDSTACK-6058 https://issues.apache.org/jira/browse/CLOUDSTACK-6058 Repository: cloudstack-git Description --- Update the java bindings that are used in cloudstack with the latest one. There are two parts for this change, 1st one is to replace the java bindings; and second part is to add the customization's made in xenapi for Cloudstack. 1. https://reviews.apache.org/r/17843/ - Replaces the java bindings. 2. https://reviews.apache.org/r/17844/ - Customizations in xenapi for CloudStack. Diffs - deps/XenServerJava/src/com/xensource/xenapi/Connection.java a652450 deps/XenServerJava/src/com/xensource/xenapi/Event.java ec43814 deps/XenServerJava/src/com/xensource/xenapi/VDI.java 992a6a6 Diff: https://reviews.apache.org/r/17844/diff/ Testing --- Thanks, Sanjay Tripathi
Re: Not able to add primary Storage Cloudstack 4.3
Hi Paul, I have just join dev-list. So didn't received your mail but I show your mail in mail-archive. I think I am not using latest system templete iso. I have downloaded system vm template from http://download.cloud.com/templates/4.3/systemvm64template-2013-12-23-hyperv.vhd.bz2 please suggest where can I download latest system vm template for hyperv . I found http://jenkins.buildacloud.org/view/4.3/job/cloudstack-4.3-systemvm/but hyperv is not available. While creating SSVM and CPVM, its not able identify any cloudastack element,except Zone id. and at some point it was searching for Xen hypervisor, though I was working with hyperv. Now I am not able to delete SSMV and CPVM, it's in Expunging state since last hour. expunge delay = 60 Regards, Tejas On Fri, Feb 7, 2014 at 5:04 PM, Paul Angus paul.an...@shapeblue.com wrote: Tejas, Check out the dev list I've just posted an issue with the system vms not starting - you might be having the same issue. check the agent log for: ERROR HypervResource.HypervResourceController [6b05182a-efa1-4105-bc5c-0a49c55674f0] - com.cloud.agent.api.StartCommand fail on exceptionObject reference not set to an instance of an object. System.NullReferenceException: Object reference not set to an instance of an object. at HypervResource.WmiCallsV2.DeployVirtualMachine(Object jsonObj, String systemVmIso) at HypervResource.HypervResourceController.StartCommand(Object cmd) Regards, Paul Angus Cloud Architect S: +44 20 3603 0540 | M: +447711418784 | T: @CloudyAngus paul.an...@shapeblue.com -Original Message- From: Tejas Gadaria [mailto:refond.g...@gmail.com] Sent: 07 February 2014 11:31 To: Rajesh Battala Cc: us...@cloudstack.apache.org; dev@cloudstack.apache.org Subject: Re: Not able to add primary Storage Cloudstack 4.3 Hi Rajesh, Thanks for providing API. I have successfully attached primary storage to cloudstack. I am facing problem with SSVM and CPVM also, probably I will initiate new mail-trail ssytem vm issues. Reards, Tejas On Thu, Feb 6, 2014 at 12:27 PM, Rajesh Battala rajesh.batt...@citrix.comwrote: Tejas, This is how the api command looks like to create primary and secondary storage. command=createStoragePoolzoneid=581bc2b6-0295-40e0-90ed-2717e239c2b6 podId=896cff2e-d51e-454b-97ed-cab078bf3be2clusterid=3785c58a-d891-448 2-9e6f-72f2060b80d3name=psscope=clusterurl=cifs%3A%2F%2F10.102.192. 150%2Fsmb-share%2Frajesh%2Fprimary%3Fuser%3DAdministrator%26domain%3DB LRresponse=jsonsessionkey=1L5raRYhxHCAgXHJ1WGvLjs7Zws%3D_=139166689 6777 command=addImageStoreresponse=jsonsessionkey=1L5raRYhxHCAgXHJ1WGvLjs 7Zws%3Dname=secprovider=SMBzoneid=581bc2b6-0295-40e0-90ed-2717e239c 2b6url=cifs%3A%2F%2F10.102.192.150%2Fsmb-share%2Frajesh%2Fsecondary%3 Fuser%3DAdministrator%26domain%3DBLR_=1391666908684 in the wizard specify the path as /smb-share/rajesh/primary Similar for secondary. Provide credentials and domain name in the wizard. Make sure you were able to access the share from the hyperv hosts before adding them to CS. Thanks Rajesh Battala *From:* Tejas Gadaria [mailto:refond.g...@gmail.com] *Sent:* Thursday, February 6, 2014 11:03 AM *To:* us...@cloudstack.apache.org *Cc:* Rajesh Battala; dev@cloudstack.apache.org *Subject:* Re: Not able to add primary Storage Cloudstack 4.3 Hi, I am not able to attach primary storage in Cloudstack as cifs. So what I am doing wrong OR what are the issues with primary storage.? Regards Tejas On Wed, Feb 5, 2014 at 4:35 PM, Tejas Gadaria refond.g...@gmail.com wrote: Hi Paul, forgot to mention, hyper-v agent is running. Regards, Tejas On Wed, Feb 5, 2014 at 4:33 PM, Tejas Gadaria refond.g...@gmail.com wrote: Hi Paul, yesterday when I was trying to add secondary storage with /hcloud/secondary , In logs I was not able to see much activity, but storage got attached. I have uploaded screenshot in previous mail. I have reattached the secondary storage, and it logs are Trying to add a new data store at cifs:// 10.129.151.55/hcloud/secondary?user=administratordomain=nwx.com to data center 3 2014-02-05 15:36:07,825 DEBUG [c.c.u.UriUtils] (catalina-exec-20:ctx-a19af0fa ctx-fb83ed9b) foundUser istrue 2014-02-05 15:36:07,825 DEBUG [c.c.u.UriUtils] (catalina-exec-20:ctx-a19af0fa ctx-fb83ed9b) foundPswd istrue I have not enable the zone yet. In attached screenshot of secondary storage in previous mail protocol:cifs provider : NFS I suspect, it because I have taken chunk of NFS datastore, mounted and share it via Windows fie server (Standalone) please find logs in attached file. Regards, Tejas On Tue, Feb 4, 2014 at 5:52 PM, Paul Angus paul.an...@shapeblue.com wrote: Hey Tejas,
RE: systemvm-4.3.0.iso
Thanks Devdeep,
@Animesh, can we make sure the patch gets into the next RC please.
Many thanks
Paul.
Regards,
Paul Angus
Cloud Architect
S: +44 20 3603 0540 | M: +447711418784 | T: @CloudyAngus
paul.an...@shapeblue.com
-Original Message-
From: Devdeep Singh [mailto:devdeep.si...@citrix.com]
Sent: 07 February 2014 12:13
To: dev@cloudstack.apache.org
Cc: Rajesh Battala
Subject: RE: systemvm-4.3.0.iso
Hi Paul,
Looks like you are using untagged vlan in your setup. There was a bug in agent
code with respect to that. Rajesh made a fix for it in commit-id
7df148c762510f669b60edff9d8d690c800b1f95 few days back. This fix should address
the problem.
Regards,
Devdeep
-Original Message-
From: Paul Angus [mailto:paul.an...@shapeblue.com]
Sent: Friday, February 7, 2014 4:55 PM
To: dev@cloudstack.apache.org
Subject: RE: systemvm-4.3.0.iso
Thanks for the info Devdeep,
so yes the systemvm iso is getting copied to the local disk, but I'm getting
the following error:
ERROR HypervResource.HypervResourceController
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - com.cloud.agent.api.StartCommand fail
on exceptionObject reference not set to an instance of an object.
System.NullReferenceException: Object reference not set to an instance of an
object.
at HypervResource.WmiCallsV2.DeployVirtualMachine(Object jsonObj, String
systemVmIso)
at HypervResource.HypervResourceController.StartCommand(Object cmd)
agent log:
2014-02-07 11:19:56,685 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Going ahead with create VM s-27-VM, 1
vcpus, 512MB RAM
2014-02-07 11:19:56,716 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Created VM s-27-VM
2014-02-07 11:19:57,075 [28] INFO HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - VM with display name s-27-VM has GUID
9E6D3606-8210-4E84-8796-BEB7970785F4
2014-02-07 11:19:57,075 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Resources for vm s-27-VM: 512 MB
memory, 1 vcpus
2014-02-07 11:19:57,106 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - New controller type
Microsoft:Hyper-V:Synthetic SCSI Controller WMI path is
\\WIN-G23HSIAIU4O\root\virtualization\v2:Msvm_ResourceAllocationSettingData.InstanceID=Microsoft:9E6D3606-8210-4E84-8796-BEB7970785F4\\57035109-ED28-4F65-8BCB-487CFC66F8E8\\0s
2014-02-07 11:19:57,106 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Going to create s-27-VM with attached
voluem ROOT-27 at \\10.0.1.27\hypervPri\stor\Primary\ROOT-27.vhd
2014-02-07 11:19:57,106 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Create disk type
Microsoft:Hyper-V:Synthetic Disk Drive (Named: ROOT-27), on vm s-27-VM ,
inserting disk\\10.0.1.27\hypervPri\stor\Primary\ROOT-27.vhd
2014-02-07 11:19:57,106 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Creating DISK for VM s-27-VM (GUID
9E6D3606-8210-4E84-8796-BEB7970785F4) by attaching
\\10.0.1.27\hypervPri\stor\Primary\ROOT-27.vhd
2014-02-07 11:19:57,403 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Creating disk drive type
Microsoft:Hyper-V:Synthetic Disk Drive, parent IDE controller is
\\WIN-G23HSIAIU4O\root\virtualization\v2:Msvm_ResourceAllocationSettingData.InstanceID=Microsoft:9E6D3606-8210-4E84-8796-BEB7970785F4\\83F8638B-8DCA-4152-9EDA-2CA8B33039B4\\0
and address on controller is 0
2014-02-07 11:19:57,418 [28] DEBUG HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - New disk drive type
Microsoft:Hyper-V:Synthetic Disk Drive WMI path is
\\WIN-G23HSIAIU4O\root\virtualization\v2:Msvm_ResourceAllocationSettingData.InstanceID=Microsoft:9E6D3606-8210-4E84-8796-BEB7970785F4\\83F8638B-8DCA-4152-9EDA-2CA8B33039B4\\0\\0\\Ds
2014-02-07 11:19:57,574 [28] INFO HypervResource.WmiCallsV2
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - Created disk
\\WIN-G23HSIAIU4O\root\virtualization\v2:Msvm_StorageAllocationSettingData.InstanceID=Microsoft:9E6D3606-8210-4E84-8796-BEB7970785F4\\83F8638B-8DCA-4152-9EDA-2CA8B33039B4\\0\\0\\L
for VM s-27-VM (GUID 9E6D3606-8210-4E84-8796-BEB7970785F4), image
\\10.0.1.27\hypervPri\stor\Primary\ROOT-27.vhd
2014-02-07 11:19:57,574 [28] ERROR HypervResource.HypervResourceController
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - com.cloud.agent.api.StartCommand fail
on exceptionObject reference not set to an instance of an object.
System.NullReferenceException: Object reference not set to an instance of an
object.
at HypervResource.WmiCallsV2.DeployVirtualMachine(Object jsonObj, String
systemVmIso)
at HypervResource.HypervResourceController.StartCommand(Object cmd)
2014-02-07 11:19:57,574 [28] INFO HypervResource.HypervResourceController
[6b05182a-efa1-4105-bc5c-0a49c55674f0] - {
com.cloud.agent.api.StartAnswer: {
result: false,
details: com.cloud.agent.api.StartCommand fail on
Review Request 17847: CLOUDSTACK-5978: Changes in get_template function to make it independent of ostype passed through test case
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17847/ --- Review request for cloudstack and Girish Shilamkar. Bugs: CLOUDSTACK-5978 https://issues.apache.org/jira/browse/CLOUDSTACK-5978 Repository: cloudstack-git Description --- Some test cases are failing because the ostype passed to the get_template function is different in each test case and it fails whenever it does not match with the ostype of template in the CS setup. Changed get_template function so that ostype id does not have any effect on the returning data (It will always return the feature, built in template in the given zone which is in READY state). This way we eliminate the dependency between the test case and the CS setup. This is the first set of change in marvin library. In next step, we can remove the hard coded ostype from all the test cases. For now, this is sufficient to save test cases from failing due to hard coded value. Diffs - tools/marvin/marvin/integration/lib/common.py 550de1a Diff: https://reviews.apache.org/r/17847/diff/ Testing --- Yes. Thanks, Ashutosh Kelkar
Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17747/#review33928 --- plugins/hypervisors/xen/src/com/cloud/hypervisor/XenServerGuru.java https://reviews.apache.org/r/17747/#comment63740 Will the upgrade path get this config ?? - Nitin Mehta On Feb. 7, 2014, 11:19 a.m., Harikrishna Patnala wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17747/ --- (Updated Feb. 7, 2014, 11:19 a.m.) Review request for cloudstack, Abhinandan Prateek, Kishan Kavala, and Nitin Mehta. Bugs: CLOUDSTACK-6023 https://issues.apache.org/jira/browse/CLOUDSTACK-6023 Repository: cloudstack-git Description --- CLOUDSTACK-6023: Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits Changed the VCPU max limit to 16 and provided a cluster level configuration parameter for this max limit named xen.vm.vcpu.max Diffs - api/src/com/cloud/agent/api/to/VirtualMachineTO.java e6240ff plugins/hypervisors/xen/src/com/cloud/hypervisor/XenServerGuru.java 28bd724 plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java 48ccf70 Diff: https://reviews.apache.org/r/17747/diff/ Testing --- Thanks, Harikrishna Patnala
Re: Code quality, QA, etc
I would love to see pre-commit testing such as what Hugo described. At the time being, I tend to mvn -P developer,systemvm clean install to make sure I have a clean build and run whatever tests it runs, then I run my own suite of tests manually (I'd like to automated these when I have time), then I check my code in. On Fri, Feb 7, 2014 at 5:02 AM, Sudha Ponnaganti sudha.ponnaga...@citrix.com wrote: +1 for pre- commit testing. Whichever tool enforces it would be good choice. For feature check in, we ( community) require sanity tests to be submitted by feature owners and this was followed well in 4.0 release but there is lapse in this practice now. This would be a great if RM can enforce this during check ins - review unit tests and results before approving a check in. -Original Message- From: Trippie [mailto:trip...@gmail.com] On Behalf Of Hugo Trippaers Sent: Friday, February 07, 2014 12:46 AM To: dev Cc: jfarr...@apache.org Subject: Re: Code quality, QA, etc Hey David, I would make a distinction between code issues and functional issues. Occasionally somebody just plainly breaks the build, i'm guilty of that myself actually, and thats just plain stupid. Luckily we have Jenkins to catch these errors quickly. I'm in a continuous struggle with Jenkins to get the build time to less than 5 minutes. I feel that is an acceptable time to get feedback on a commit, any longer and you have moved on to the next thing or gone home. Also this kind of testing isn't really hard, run the build and unit tests. By introducing something like gerrit we can actually make this happen before committing it to the repo. Push a patch to gerrit, gerrit tells jenkins to test the patch, if +1 from jerkins commit, for non committers the step would be to invite somebody for review as well. Second nice thing about jenkins is the post-review test, if a contributor submits a patch its build by jenkins, if a reviewes approves the patch, jerkins will again run a build to ensure that the patch will still apply and doesn't break the build. Very handy if there is some time between patch submission and patch review. Functional issues are much harder to track. For example yesterday i found several issues in the contrail plugin that would not cause any pain in a contrail environment, but any other environments creating a network would fail. These examples are too common and difficult to catch with unit tests. It can be done, but requires some serious effort on the developers side and we in general don't seem to be very active at writing unit tests. These kind of issues can only be found by actually running CloudStack and executing a series of functional tests. Ideally that is what we have the BVT suite for, but i think our current BVT setup is not documented enough to give accurate feedback to a developer about which patch broke a certain piece of functionality. In jenkins the path from code to BVT is not kept yet, so it is almost impossible to see which commits were new in a particular run of the bvt suite. Personally i'm trying to get into the habit of running a series of tests on devcloud before committing something. Doesn't prove a lot, but does guarantee that the bare basic developer functionality is working before committing something. After a commit at least i'm sure that anybody will be able to spinup devcloud and deploy an instance. I'm trying to get this automated as well so we can use this as feedback on a patch. Beers for anyone who writes an easy to use script that configures devcloud with a zone and tests if a user vm can be instantiated on an isolated sourcenat network. If we could include such a script in the tree it might help people with testing their patch before committing. I think we are seeing more and more reverts in the tree. Not necessarily a good thing, but at least people know that there is that option if a commit really breaks a build. Also please help each other out, everybody can make a mistake and commit it. If its a trivial mistake it might not be much effort to track it down and fix it, which is way better than a revert or a mail that something is broken. In short, we need to make testing more efficient and transparent to allow people to easily incorporate it in their personal workflow. Cheers, Hugo On 7 feb. 2014, at 04:50, David Nalley da...@gnsa.us wrote: Hi folks, We continue to break things large and small in the codebase, and after a number of different conversations; I thought I'd bring that discussion here. First - coding quality is only one factor that the PMC considers when making someone a committer. Second - CloudStack is a huge codebase; has a ton of inter-related pieces, and unintended consequences are easy. We also have an pretty heady commit velocity - 20+ commits today alone. Some communities have Review-then-commit - which would slow us down, and presumably help us increase
BUG report: Excessive use of LVM snapshots on XenServer, that leads to snapshot failure and unnecessary disk usage.
Hi guys, is there a chance that someone looks at my bug report, to check if it does make sense? https://issues.apache.org/jira/browse/CLOUDSTACK-6060 I've also noticed that my older bug reports form half a year ago, have had no status update. Is this normal and expected? Is it possible that they get overlooked? Regards, F.
Re: [PROPOSAL] load balancing configuration for VM nic secondary ips
+1. Any UI changes? On 2/6/14 5:11 AM, Jayapal Reddy Uradi jayapalreddy.ur...@citrix.com wrote: Hi, Multiple ips per nic feature allows user to acquire more ip address on vm nic. On these ip addresses user able to configure PF and static NAT. With this feature user can also configure the load balancing rules for vm secondary ips also. Please review the below and give your comments. https://cwiki.apache.org/confluence/display/CLOUDSTACK/Configuring+load+ba lancing+rules+for+VM+nic+secondary+ips Ticket id: CLOUDSTACK-2692 Thanks, Jayapal
Re: [PROPOSAL] List VM API enhancement
Hi Koushik, I agree with the idea of supporting multiple IDs. But I may not like the idea of introducing another different query parameter ids for this purpose. Why cannot we just change current id parameter to take a list of values? This way, user will not need to use two different parameters for single or multiple cases. Maintaining two different parameters for similar purpose is error-prone. If you look at Amazon EC2 api, you will notice that they are also using the similar convention, id parameter can be one or more. Thanks -min On 2/6/14 3:24 AM, Koushik Das koushik@citrix.com wrote: Yes it will be like a findByIds() and the one id case is just a special case for this. On 06-Feb-2014, at 4:24 PM, Daan Hoogland daan.hoogl...@gmail.com wrote: looks nice, it will be backed by the current query for one id? or will you write a findByIds()? On Thu, Feb 6, 2014 at 9:35 AM, Abhinandan Prateek abhinandan.prat...@citrix.com wrote: +1, The listVM call is one of the most resource intensive call. Any step to optimise it are welcome. On 06/02/14 2:01 pm, Koushik Das koushik@citrix.com wrote: Currently list VM can only be called using a single VM ID. So if there is a need to query a set of VMs using ID then either multiple list VM calls need to be made or all VMs needs to be fetched and then do a client side filtering. Both approaches are sub-optimal - the former results in multiple queries to database and the latter will be an overkill if you need a small subset from a very large number of VMs. The proposal is to have an additional parameter to specify a list of VM IDs for which the data needs to be fetched. Using this the required VMs can be queried in an efficient manner. With the new parameter the syntax would look like http://localhost:8096/api?command=listVirtualMachineslistAll=trueids= edd ac053-9b12-4d2e-acb7-233de2e98112,009966fc-4d7b-4f84-8609-254979ba0134 The new 'ids' parameter will be mutually exclusive with the existing 'id' parameter. Let me know if there are any concerns/comments. Thanks, Koushik -- Daan
Re: commit 6523c068695d0431070060667c222eb40d54b14d breaks network removal
Daan,
1) What is the reason you execute this code snippet just for Shared
networks?
2) As I suggested in my prev email, before retrieving Dhcpprovider, you
should check if dhcp service is enabled on the network. Use method
areServicesSupportedInNetwork
From NetworkModel to check that.
-Alena.
On 2/6/14, 10:04 PM, Daan Hoogland daan.hoogl...@gmail.com wrote:
Alena,
The revert didn't apply. Would the folowing do the trick?
if (vm.getType() == Type.User
network.getTrafficType() == TrafficType.Guest
network.getGuestType() == GuestType.Shared) {
// remove the dhcpservice ip if this is the last nic in
subnet.
DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
if (dhcpServiceProvider != null
isDhcpAccrossMultipleSubnetsSupported(dhcpServiceProvider)
isLastNicInSubnet(nic)) {
removeDhcpServiceInSubnet(nic);
}
}
On Fri, Feb 7, 2014 at 6:55 AM, Daan Hoogland daan.hoogl...@gmail.com
wrote:
second thought,
Soheils mail bounces and the commit does not refer a ticket from jira.
I am going to revert. I should have been more vigilant. sorry.
On Fri, Feb 7, 2014 at 6:49 AM, Daan Hoogland daan.hoogl...@gmail.com
wrote:
will do Alena,
thanks for the headsup
On Thu, Feb 6, 2014 at 10:42 PM, Alena Prokharchyk
alena.prokharc...@citrix.com wrote:
Soheil/Daan,
The commit in the subject breaks network System vms destroy (VR, SSVM,
CPVM), resulting in the network removal failures. Following line
replacement
causes the failure:
- if (vm.getType() == Type.User
isDhcpAccrossMultipleSubnetsSupported(network)
isLastNicInSubnet(nic)
network.getTrafficType() == TrafficType.Guest
With
+DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
When you try to call getDhcpServiceProvider(network), it throws an
exception
because DHCP service is not enabled in Public/Control networks of
system vms
nics. So system vm always fails to expunge.
Could you please fix it by checking if DHCP service is enabled on the
network, before getting the DHCP service provider?
Thanks,
Alena.
--
Daan
--
Daan
--
Daan
DB upgrade 4.2.1 - 4.3 fails
Just tested this today, it happens consistently in my setup. Opened https://issues.apache.org/jira/browse/CLOUDSTACK-6061 with plenty of details. -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro
Re: commit 6523c068695d0431070060667c222eb40d54b14d breaks network removal
H Alena,
I am just trying to fix an old contribution that I applied as it
seemed not to harm in a basic test. revert didn't work so I am looking
for a quick remedy. The original patch does it for shared only. I
don't care either way. Lets do the best thing.
the code now
if (vm.getType() == Type.User
_networkModel.areServicesSupportedInNetwork(network.getId(),
Service.Dhcp)
network.getTrafficType() == TrafficType.Guest
network.getGuestType() == GuestType.Shared) {
// remove the dhcpservice ip if this is the last nic in subnet.
DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
if (dhcpServiceProvider != null
isDhcpAccrossMultipleSubnetsSupported(dhcpServiceProvider)
isLastNicInSubnet(nic)) {
removeDhcpServiceInSubnet(nic);
}
}
What do you sugest?
if (vm.getType() == Type.User
_networkModel.areServicesSupportedInNetwork(network.getId(),
Service.Dhcp)) {
// remove the dhcpservice ip if this is the last nic in subnet.
DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
if (dhcpServiceProvider != null
isDhcpAccrossMultipleSubnetsSupported(dhcpServiceProvider)
isLastNicInSubnet(nic)
network.getTrafficType() == TrafficType.Guest
network.getGuestType() == GuestType.Shared) {
removeDhcpServiceInSubnet(nic);
}
}
???
On Fri, Feb 7, 2014 at 6:56 PM, Alena Prokharchyk
alena.prokharc...@citrix.com wrote:
Daan,
1) What is the reason you execute this code snippet just for Shared
networks?
2) As I suggested in my prev email, before retrieving Dhcpprovider, you
should check if dhcp service is enabled on the network. Use method
areServicesSupportedInNetwork
From NetworkModel to check that.
-Alena.
On 2/6/14, 10:04 PM, Daan Hoogland daan.hoogl...@gmail.com wrote:
Alena,
The revert didn't apply. Would the folowing do the trick?
if (vm.getType() == Type.User
network.getTrafficType() == TrafficType.Guest
network.getGuestType() == GuestType.Shared) {
// remove the dhcpservice ip if this is the last nic in
subnet.
DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
if (dhcpServiceProvider != null
isDhcpAccrossMultipleSubnetsSupported(dhcpServiceProvider)
isLastNicInSubnet(nic)) {
removeDhcpServiceInSubnet(nic);
}
}
On Fri, Feb 7, 2014 at 6:55 AM, Daan Hoogland daan.hoogl...@gmail.com
wrote:
second thought,
Soheils mail bounces and the commit does not refer a ticket from jira.
I am going to revert. I should have been more vigilant. sorry.
On Fri, Feb 7, 2014 at 6:49 AM, Daan Hoogland daan.hoogl...@gmail.com
wrote:
will do Alena,
thanks for the headsup
On Thu, Feb 6, 2014 at 10:42 PM, Alena Prokharchyk
alena.prokharc...@citrix.com wrote:
Soheil/Daan,
The commit in the subject breaks network System vms destroy (VR, SSVM,
CPVM), resulting in the network removal failures. Following line
replacement
causes the failure:
- if (vm.getType() == Type.User
isDhcpAccrossMultipleSubnetsSupported(network)
isLastNicInSubnet(nic)
network.getTrafficType() == TrafficType.Guest
With
+DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
When you try to call getDhcpServiceProvider(network), it throws an
exception
because DHCP service is not enabled in Public/Control networks of
system vms
nics. So system vm always fails to expunge.
Could you please fix it by checking if DHCP service is enabled on the
network, before getting the DHCP service provider?
Thanks,
Alena.
--
Daan
--
Daan
--
Daan
--
Daan
RE: [PROPOSAL] List VM API enhancement
+1 It's confusing to have id and ids all over the place. We should just say all ids can come in arrays by default. --Alex -Original Message- From: Min Chen [mailto:min.c...@citrix.com] Sent: Friday, February 7, 2014 9:56 AM To: dev@cloudstack.apache.org Subject: Re: [PROPOSAL] List VM API enhancement Hi Koushik, I agree with the idea of supporting multiple IDs. But I may not like the idea of introducing another different query parameter ids for this purpose. Why cannot we just change current id parameter to take a list of values? This way, user will not need to use two different parameters for single or multiple cases. Maintaining two different parameters for similar purpose is error-prone. If you look at Amazon EC2 api, you will notice that they are also using the similar convention, id parameter can be one or more. Thanks -min On 2/6/14 3:24 AM, Koushik Das koushik@citrix.com wrote: Yes it will be like a findByIds() and the one id case is just a special case for this. On 06-Feb-2014, at 4:24 PM, Daan Hoogland daan.hoogl...@gmail.com wrote: looks nice, it will be backed by the current query for one id? or will you write a findByIds()? On Thu, Feb 6, 2014 at 9:35 AM, Abhinandan Prateek abhinandan.prat...@citrix.com wrote: +1, The listVM call is one of the most resource intensive call. Any step to optimise it are welcome. On 06/02/14 2:01 pm, Koushik Das koushik@citrix.com wrote: Currently list VM can only be called using a single VM ID. So if there is a need to query a set of VMs using ID then either multiple list VM calls need to be made or all VMs needs to be fetched and then do a client side filtering. Both approaches are sub-optimal - the former results in multiple queries to database and the latter will be an overkill if you need a small subset from a very large number of VMs. The proposal is to have an additional parameter to specify a list of VM IDs for which the data needs to be fetched. Using this the required VMs can be queried in an efficient manner. With the new parameter the syntax would look like http://localhost:8096/api?command=listVirtualMachineslistAll=truei ds= edd ac053-9b12-4d2e-acb7-233de2e98112,009966fc-4d7b-4f84-8609- 254979ba01 34 The new 'ids' parameter will be mutually exclusive with the existing 'id' parameter. Let me know if there are any concerns/comments. Thanks, Koushik -- Daan
Re: Source build failing in DevCloud
Hi Suresh, If you struggle to make progress, you could take a look at https://github.com/snowch/devcloud I'm hoping that this environment will be a replacement for devcloud2. It also checks out 4.3 and builds it. Please report any bugs on the github issue tracker. Many thanks, Chris On Fri, Feb 7, 2014 at 12:05 PM, CK cloudw...@gmail.com wrote: Java version is: javac 1.6.0_24 I'm new to 'git', how do I ...checkout the 4.3 branch, git pull then compile you should get the 4.3 current build., can you provide the commands? Thanks On 7 February 2014 12:00, Alex Hitchins alex.hitch...@shapeblue.com wrote: I've had a quick look at the error statements - what version of Java is on the devcloud vm? Also, if you checkout the 4.3 branch, git pull then compile you should get the 4.3 current build. Regards Alex Hitchins D: +44 1892 523 587 | S: +44 20 3603 0540 | M: +44 7788 423 969 alex.hitch...@shapeblue.com -Original Message- From: CK [mailto:cloudw...@gmail.com] Sent: 07 February 2014 11:41 To: dev@cloudstack.apache.org Subject: Source build failing in DevCloud Hi, This is my first attempt at building CloudStack from the source code and decided on using DevCloud. I followed the wiki for using DevCloud https://cwiki.apache.org/confluence/display/CLOUDSTACK/DevCloud. I want to build the 4.3 release and used the updated DevCloud 2.0 appliance and as per the instructions. I git cloned the latest master code, but when I then run mvn -P developer,systemvm clean install. I get a compilation/build failure (see output below). Q1) Have I missed something and why I am I getting this error? Q2) I see it is building 4.4.0-SNAPSHOT how do I get it to build 4.3? Thanks. [INFO] [INFO] Building Apache CloudStack Developer Tools - Checkstyle Configuration 4.4.0-SNAPSHOT [INFO] [INFO] [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ checkstyle --- [INFO] Deleting /opt/cloudstack/cloudstack/tools/checkstyle/target [INFO] [INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ checkstyle --- [debug] execute contextualize [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] Copying 1 resource [INFO] [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ checkstyle --- [INFO] No sources to compile [INFO] [INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ checkstyle --- [debug] execute contextualize [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory /opt/cloudstack/cloudstack/tools/checkstyle/src/test/resources [INFO] [INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ checkstyle --- [INFO] No sources to compile [INFO] [INFO] --- maven-surefire-plugin:2.10:test (default-test) @ checkstyle --- [INFO] No tests to run. [INFO] Surefire report directory: /opt/cloudstack/cloudstack/tools/checkstyle/target/surefire-reports --- T E S T S --- Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] --- maven-jar-plugin:2.3.2:jar (default-jar) @ checkstyle --- [INFO] Building jar: /opt/cloudstack/cloudstack/tools/checkstyle/target/checkstyle-4.4.0-SNAPSHOT.jar [INFO] [INFO] --- maven-install-plugin:2.3.1:install (default-install) @ checkstyle --- [INFO] Installing /opt/cloudstack/cloudstack/tools/checkstyle/target/checkstyle-4.4.0-SNAPSHOT.jar to /root/.m2/repository/org/apache/cloudstack/checkstyle/4.4.0-SNAPSHOT/checkstyle-4.4.0-SNAPSHOT.jar [INFO] Installing /opt/cloudstack/cloudstack/tools/checkstyle/pom.xml to /root/.m2/repository/org/apache/cloudstack/checkstyle/4.4.0-SNAPSHOT/checkstyle-4.4.0-SNAPSHOT.pom [INFO] [INFO] [INFO] Building Apache CloudStack 4.4.0-SNAPSHOT [INFO] [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloudstack --- [INFO] Deleting /opt/cloudstack/cloudstack/target (includes = [**/*], excludes = []) [INFO] Deleting /opt/cloudstack/cloudstack (includes = [target, dist], excludes = []) [INFO] [INFO] --- maven-remote-resources-plugin:1.3:process (default) @ cloudstack --- [INFO] [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ cloudstack --- [INFO] [INFO] --- maven-install-plugin:2.3.1:install (default-install) @ cloudstack --- [INFO] Installing /opt/cloudstack/cloudstack/pom.xml to /root/.m2/repository/org/apache/cloudstack/cloudstack/4.4.0-SNAPSHOT/cloudstack-4.4.0-SNAPSHOT.pom [INFO]
Re: commit 6523c068695d0431070060667c222eb40d54b14d breaks network removal
Daan,
Here is how it should look:
//1) Make all the checks that used to exist in original code + if DHCP
service is enabled on the network
if (vm.getType() == Type.User network.getTrafficType() ==
TrafficType.Guest isLastNicInSubnet(nic) network.getGuestType() ==
GuestType.Shared
_networkModel.areServicesSupportedInNetwork(network.getId(),Service.Dhcp))
{
//2) Now get the DHCP provider, and do the rest of the checks
DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
if (dhcpServiceProvider != null
isDhcpAccrossMultipleSubnetsSupported(dhcpServiceProvider)) {
removeDhcpServiceInSubnet(nic);
}
}
Could you please test it for 2 Shared networks - one with DHCP service,
and one w/o?
Thank you!
Alena.
On 2/7/14, 10:04 AM, Daan Hoogland daan.hoogl...@gmail.com wrote:
H Alena,
I am just trying to fix an old contribution that I applied as it
seemed not to harm in a basic test. revert didn't work so I am looking
for a quick remedy. The original patch does it for shared only. I
don't care either way. Lets do the best thing.
the code now
if (vm.getType() == Type.User
_networkModel.areServicesSupportedInNetwork(network.getId(),
Service.Dhcp)
network.getTrafficType() == TrafficType.Guest
network.getGuestType() == GuestType.Shared) {
// remove the dhcpservice ip if this is the last nic in
subnet.
DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
if (dhcpServiceProvider != null
isDhcpAccrossMultipleSubnetsSupported(dhcpServiceProvider)
isLastNicInSubnet(nic)) {
removeDhcpServiceInSubnet(nic);
}
}
What do you sugest?
if (vm.getType() == Type.User
_networkModel.areServicesSupportedInNetwork(network.getId(),
Service.Dhcp)) {
// remove the dhcpservice ip if this is the last nic in
subnet.
DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
if (dhcpServiceProvider != null
isDhcpAccrossMultipleSubnetsSupported(dhcpServiceProvider)
isLastNicInSubnet(nic)
network.getTrafficType() == TrafficType.Guest
network.getGuestType() == GuestType.Shared) {
removeDhcpServiceInSubnet(nic);
}
}
???
On Fri, Feb 7, 2014 at 6:56 PM, Alena Prokharchyk
alena.prokharc...@citrix.com wrote:
Daan,
1) What is the reason you execute this code snippet just for Shared
networks?
2) As I suggested in my prev email, before retrieving Dhcpprovider, you
should check if dhcp service is enabled on the network. Use method
areServicesSupportedInNetwork
From NetworkModel to check that.
-Alena.
On 2/6/14, 10:04 PM, Daan Hoogland daan.hoogl...@gmail.com wrote:
Alena,
The revert didn't apply. Would the folowing do the trick?
if (vm.getType() == Type.User
network.getTrafficType() == TrafficType.Guest
network.getGuestType() == GuestType.Shared) {
// remove the dhcpservice ip if this is the last nic in
subnet.
DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
if (dhcpServiceProvider != null
isDhcpAccrossMultipleSubnetsSupported(dhcpServiceProvider)
isLastNicInSubnet(nic)) {
removeDhcpServiceInSubnet(nic);
}
}
On Fri, Feb 7, 2014 at 6:55 AM, Daan Hoogland daan.hoogl...@gmail.com
wrote:
second thought,
Soheils mail bounces and the commit does not refer a ticket from jira.
I am going to revert. I should have been more vigilant. sorry.
On Fri, Feb 7, 2014 at 6:49 AM, Daan Hoogland
daan.hoogl...@gmail.com
wrote:
will do Alena,
thanks for the headsup
On Thu, Feb 6, 2014 at 10:42 PM, Alena Prokharchyk
alena.prokharc...@citrix.com wrote:
Soheil/Daan,
The commit in the subject breaks network System vms destroy (VR,
SSVM,
CPVM), resulting in the network removal failures. Following line
replacement
causes the failure:
- if (vm.getType() == Type.User
isDhcpAccrossMultipleSubnetsSupported(network)
isLastNicInSubnet(nic)
network.getTrafficType() == TrafficType.Guest
With
+DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
When you try to call getDhcpServiceProvider(network), it throws an
exception
because DHCP service is not enabled in Public/Control networks of
system vms
nics. So system vm always fails to expunge.
Could you please fix it by checking if DHCP service is enabled on
the
network, before getting the DHCP service provider?
Thanks,
Alena.
--
Daan
--
Daan
--
Daan
--
Daan
RE: [PROPOSAL] List VM API enhancement
Yes, agree to this as well. Accordingly we need to handle the getEntityOwnerId() dependency. -Original Message- From: Alex Huang [mailto:alex.hu...@citrix.com] Sent: Friday, February 07, 2014 10:06 AM To: dev@cloudstack.apache.org Subject: RE: [PROPOSAL] List VM API enhancement +1 It's confusing to have id and ids all over the place. We should just say all ids can come in arrays by default. --Alex -Original Message- From: Min Chen [mailto:min.c...@citrix.com] Sent: Friday, February 7, 2014 9:56 AM To: dev@cloudstack.apache.org Subject: Re: [PROPOSAL] List VM API enhancement Hi Koushik, I agree with the idea of supporting multiple IDs. But I may not like the idea of introducing another different query parameter ids for this purpose. Why cannot we just change current id parameter to take a list of values? This way, user will not need to use two different parameters for single or multiple cases. Maintaining two different parameters for similar purpose is error-prone. If you look at Amazon EC2 api, you will notice that they are also using the similar convention, id parameter can be one or more. Thanks -min On 2/6/14 3:24 AM, Koushik Das koushik@citrix.com wrote: Yes it will be like a findByIds() and the one id case is just a special case for this. On 06-Feb-2014, at 4:24 PM, Daan Hoogland daan.hoogl...@gmail.com wrote: looks nice, it will be backed by the current query for one id? or will you write a findByIds()? On Thu, Feb 6, 2014 at 9:35 AM, Abhinandan Prateek abhinandan.prat...@citrix.com wrote: +1, The listVM call is one of the most resource intensive call. +Any step to optimise it are welcome. On 06/02/14 2:01 pm, Koushik Das koushik@citrix.com wrote: Currently list VM can only be called using a single VM ID. So if there is a need to query a set of VMs using ID then either multiple list VM calls need to be made or all VMs needs to be fetched and then do a client side filtering. Both approaches are sub-optimal - the former results in multiple queries to database and the latter will be an overkill if you need a small subset from a very large number of VMs. The proposal is to have an additional parameter to specify a list of VM IDs for which the data needs to be fetched. Using this the required VMs can be queried in an efficient manner. With the new parameter the syntax would look like http://localhost:8096/api?command=listVirtualMachineslistAll=true i ds= edd ac053-9b12-4d2e-acb7-233de2e98112,009966fc-4d7b-4f84-8609- 254979ba01 34 The new 'ids' parameter will be mutually exclusive with the existing 'id' parameter. Let me know if there are any concerns/comments. Thanks, Koushik -- Daan
RE: DB upgrade 4.2.1 - 4.3 fails
Alena do you mind checking into this issue? Animesh -Original Message- From: Nux! [mailto:n...@li.nux.ro] Sent: Friday, February 07, 2014 10:00 AM To: dev@cloudstack.apache.org Subject: DB upgrade 4.2.1 - 4.3 fails Just tested this today, it happens consistently in my setup. Opened https://issues.apache.org/jira/browse/CLOUDSTACK-6061 with plenty of details. -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro
Re: DB upgrade 4.2.1 - 4.3 fails
Updated the bug. Most likely happened because the upgrade was performed twice (failed the first time, and the person doing the upgrade,didn¹t do the rollback). See more details in the bug. -Alena. On 2/7/14, 10:48 AM, Animesh Chaturvedi animesh.chaturv...@citrix.com wrote: Alena do you mind checking into this issue? Animesh -Original Message- From: Nux! [mailto:n...@li.nux.ro] Sent: Friday, February 07, 2014 10:00 AM To: dev@cloudstack.apache.org Subject: DB upgrade 4.2.1 - 4.3 fails Just tested this today, it happens consistently in my setup. Opened https://issues.apache.org/jira/browse/CLOUDSTACK-6061 with plenty of details. -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro
RE: DB upgrade 4.2.1 - 4.3 fails
Nux you had a similar issue for previous RC https://issues.apache.org/jira/browse/CLOUDSTACK-5902 which was resolved as not a problem -Original Message- From: Animesh Chaturvedi [mailto:animesh.chaturv...@citrix.com] Sent: Friday, February 07, 2014 10:49 AM To: dev@cloudstack.apache.org Cc: Alena Prokharchyk Subject: RE: DB upgrade 4.2.1 - 4.3 fails Alena do you mind checking into this issue? Animesh -Original Message- From: Nux! [mailto:n...@li.nux.ro] Sent: Friday, February 07, 2014 10:00 AM To: dev@cloudstack.apache.org Subject: DB upgrade 4.2.1 - 4.3 fails Just tested this today, it happens consistently in my setup. Opened https://issues.apache.org/jira/browse/CLOUDSTACK-6061 with plenty of details. -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro
Re: [PROPOSAL] List VM API enhancement
Good point Min. I also thought about it but looking at some of the existing APIs thought of keeping both. For e.g. in deploy VM api there is a parameter called 'networkids' which can take an array of network IDs. Note that the naming convention of ending in 's'. Now by this logic we should name the parameter 'ids' and remove the existing parameter 'id' which will be a breaking change. In case the existing 'id' parameter is used for multiple IDs that breaks the parameter naming convention. I am all in favour of using the existing 'id' parameter if there is no issues with breaking the naming convention. On 07-Feb-2014, at 11:25 PM, Min Chen min.c...@citrix.com wrote: Hi Koushik, I agree with the idea of supporting multiple IDs. But I may not like the idea of introducing another different query parameter ids for this purpose. Why cannot we just change current id parameter to take a list of values? This way, user will not need to use two different parameters for single or multiple cases. Maintaining two different parameters for similar purpose is error-prone. If you look at Amazon EC2 api, you will notice that they are also using the similar convention, id parameter can be one or more. Thanks -min On 2/6/14 3:24 AM, Koushik Das koushik@citrix.com wrote: Yes it will be like a findByIds() and the one id case is just a special case for this. On 06-Feb-2014, at 4:24 PM, Daan Hoogland daan.hoogl...@gmail.com wrote: looks nice, it will be backed by the current query for one id? or will you write a findByIds()? On Thu, Feb 6, 2014 at 9:35 AM, Abhinandan Prateek abhinandan.prat...@citrix.com wrote: +1, The listVM call is one of the most resource intensive call. Any step to optimise it are welcome. On 06/02/14 2:01 pm, Koushik Das koushik@citrix.com wrote: Currently list VM can only be called using a single VM ID. So if there is a need to query a set of VMs using ID then either multiple list VM calls need to be made or all VMs needs to be fetched and then do a client side filtering. Both approaches are sub-optimal - the former results in multiple queries to database and the latter will be an overkill if you need a small subset from a very large number of VMs. The proposal is to have an additional parameter to specify a list of VM IDs for which the data needs to be fetched. Using this the required VMs can be queried in an efficient manner. With the new parameter the syntax would look like http://localhost:8096/api?command=listVirtualMachineslistAll=trueids= edd ac053-9b12-4d2e-acb7-233de2e98112,009966fc-4d7b-4f84-8609-254979ba0134 The new 'ids' parameter will be mutually exclusive with the existing 'id' parameter. Let me know if there are any concerns/comments. Thanks, Koushik -- Daan
Re: [PROPOSAL] List VM API enhancement
We can just agree from now on to use the ³id for handling multiple ids. And of course, we can never delete the ³ID² parameter just to satisfy the old convention, as this is the most used parameter :) I can see that several existing commands - archive/deleteAlerts are using ApiConstants.IDs parameter. We can mark IDs as deprecated, so its no longer used by new commands. -Alena. On 2/7/14, 11:03 AM, Koushik Das koushik@citrix.com wrote: Good point Min. I also thought about it but looking at some of the existing APIs thought of keeping both. For e.g. in deploy VM api there is a parameter called 'networkids' which can take an array of network IDs. Note that the naming convention of ending in 's'. Now by this logic we should name the parameter 'ids' and remove the existing parameter 'id' which will be a breaking change. In case the existing 'id' parameter is used for multiple IDs that breaks the parameter naming convention. I am all in favour of using the existing 'id' parameter if there is no issues with breaking the naming convention. On 07-Feb-2014, at 11:25 PM, Min Chen min.c...@citrix.com wrote: Hi Koushik, I agree with the idea of supporting multiple IDs. But I may not like the idea of introducing another different query parameter ids for this purpose. Why cannot we just change current id parameter to take a list of values? This way, user will not need to use two different parameters for single or multiple cases. Maintaining two different parameters for similar purpose is error-prone. If you look at Amazon EC2 api, you will notice that they are also using the similar convention, id parameter can be one or more. Thanks -min On 2/6/14 3:24 AM, Koushik Das koushik@citrix.com wrote: Yes it will be like a findByIds() and the one id case is just a special case for this. On 06-Feb-2014, at 4:24 PM, Daan Hoogland daan.hoogl...@gmail.com wrote: looks nice, it will be backed by the current query for one id? or will you write a findByIds()? On Thu, Feb 6, 2014 at 9:35 AM, Abhinandan Prateek abhinandan.prat...@citrix.com wrote: +1, The listVM call is one of the most resource intensive call. Any step to optimise it are welcome. On 06/02/14 2:01 pm, Koushik Das koushik@citrix.com wrote: Currently list VM can only be called using a single VM ID. So if there is a need to query a set of VMs using ID then either multiple list VM calls need to be made or all VMs needs to be fetched and then do a client side filtering. Both approaches are sub-optimal - the former results in multiple queries to database and the latter will be an overkill if you need a small subset from a very large number of VMs. The proposal is to have an additional parameter to specify a list of VM IDs for which the data needs to be fetched. Using this the required VMs can be queried in an efficient manner. With the new parameter the syntax would look like http://localhost:8096/api?command=listVirtualMachineslistAll=trueid s= edd ac053-9b12-4d2e-acb7-233de2e98112,009966fc-4d7b-4f84-8609-254979ba013 4 The new 'ids' parameter will be mutually exclusive with the existing 'id' parameter. Let me know if there are any concerns/comments. Thanks, Koushik -- Daan
Re: Code quality, QA, etc
+1 for pre-commit testing. I put pre-commit testing in practice personally, and it helped me a lot. Before I submit a patch, I create a pull request in my github repo. My Jenkins subscribes pull requests by Jenkins plugin for Github and the build result is displayed in Github. I heard there are plugins for ReviewBoard and plugins for Gerrit that automatically test patches submitted. ex. ) https://wiki.jenkins-ci.org/display/JENKINS/Jenkins-Reviewbot https://wiki.jenkins-ci.org/display/JENKINS/Gerrit+Trigger If possible, executing automated end-to-end testing is better to eliminate most of regressions, but even executing build and unit testing will prevent minor mistakes and broken commits. 2014-02-07 9:28 GMT-07:00 Mike Tutkowski mike.tutkow...@solidfire.com: I would love to see pre-commit testing such as what Hugo described. At the time being, I tend to mvn -P developer,systemvm clean install to make sure I have a clean build and run whatever tests it runs, then I run my own suite of tests manually (I'd like to automated these when I have time), then I check my code in. On Fri, Feb 7, 2014 at 5:02 AM, Sudha Ponnaganti sudha.ponnaga...@citrix.com wrote: +1 for pre- commit testing. Whichever tool enforces it would be good choice. For feature check in, we ( community) require sanity tests to be submitted by feature owners and this was followed well in 4.0 release but there is lapse in this practice now. This would be a great if RM can enforce this during check ins - review unit tests and results before approving a check in. -Original Message- From: Trippie [mailto:trip...@gmail.com] On Behalf Of Hugo Trippaers Sent: Friday, February 07, 2014 12:46 AM To: dev Cc: jfarr...@apache.org Subject: Re: Code quality, QA, etc Hey David, I would make a distinction between code issues and functional issues. Occasionally somebody just plainly breaks the build, i'm guilty of that myself actually, and thats just plain stupid. Luckily we have Jenkins to catch these errors quickly. I'm in a continuous struggle with Jenkins to get the build time to less than 5 minutes. I feel that is an acceptable time to get feedback on a commit, any longer and you have moved on to the next thing or gone home. Also this kind of testing isn't really hard, run the build and unit tests. By introducing something like gerrit we can actually make this happen before committing it to the repo. Push a patch to gerrit, gerrit tells jenkins to test the patch, if +1 from jerkins commit, for non committers the step would be to invite somebody for review as well. Second nice thing about jenkins is the post-review test, if a contributor submits a patch its build by jenkins, if a reviewes approves the patch, jerkins will again run a build to ensure that the patch will still apply and doesn't break the build. Very handy if there is some time between patch submission and patch review. Functional issues are much harder to track. For example yesterday i found several issues in the contrail plugin that would not cause any pain in a contrail environment, but any other environments creating a network would fail. These examples are too common and difficult to catch with unit tests. It can be done, but requires some serious effort on the developers side and we in general don't seem to be very active at writing unit tests. These kind of issues can only be found by actually running CloudStack and executing a series of functional tests. Ideally that is what we have the BVT suite for, but i think our current BVT setup is not documented enough to give accurate feedback to a developer about which patch broke a certain piece of functionality. In jenkins the path from code to BVT is not kept yet, so it is almost impossible to see which commits were new in a particular run of the bvt suite. Personally i'm trying to get into the habit of running a series of tests on devcloud before committing something. Doesn't prove a lot, but does guarantee that the bare basic developer functionality is working before committing something. After a commit at least i'm sure that anybody will be able to spinup devcloud and deploy an instance. I'm trying to get this automated as well so we can use this as feedback on a patch. Beers for anyone who writes an easy to use script that configures devcloud with a zone and tests if a user vm can be instantiated on an isolated sourcenat network. If we could include such a script in the tree it might help people with testing their patch before committing. I think we are seeing more and more reverts in the tree. Not necessarily a good thing, but at least people know that there is that option if a commit really breaks a build. Also please help each other out, everybody can make a mistake and commit it. If its a trivial mistake it might not be much effort to track it down and fix it, which is way better than a revert or a mail that something is broken. In short,
Re: [PROPOSAL] List VM API enhancement
I looked at BaseListCmd.java and see the following.
Prachi, Are you referring to this or anything else?
public long getEntityOwnerId() {
// no owner is needed for list command
return 0;
}
On 08-Feb-2014, at 12:02 AM, Prachi Damle
prachi.da...@citrix.commailto:prachi.da...@citrix.com wrote:
Yes, agree to this as well.
Accordingly we need to handle the getEntityOwnerId() dependency.
-Original Message-
From: Alex Huang [mailto:alex.hu...@citrix.comhttp://citrix.com]
Sent: Friday, February 07, 2014 10:06 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: RE: [PROPOSAL] List VM API enhancement
+1 It's confusing to have id and ids all over the place. We should just say
all ids can come in arrays by default.
--Alex
-Original Message-
From: Min Chen [mailto:min.c...@citrix.comhttp://citrix.com]
Sent: Friday, February 7, 2014 9:56 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: [PROPOSAL] List VM API enhancement
Hi Koushik,
I agree with the idea of supporting multiple IDs. But I may not like
the idea of introducing another different query parameter ids for this
purpose.
Why cannot we just change current id parameter to take a list of values?
This way, user will not need to use two different parameters for
single or multiple cases. Maintaining two different parameters for
similar purpose is error-prone. If you look at Amazon EC2 api, you
will notice that they are also using the similar convention, id parameter can
be one or more.
Thanks
-min
On 2/6/14 3:24 AM, Koushik Das
koushik@citrix.commailto:koushik@citrix.com wrote:
Yes it will be like a findByIds() and the one id case is just a
special case for this.
On 06-Feb-2014, at 4:24 PM, Daan Hoogland
daan.hoogl...@gmail.commailto:daan.hoogl...@gmail.com
wrote:
looks nice, it will be backed by the current query for one id? or
will you write a findByIds()?
On Thu, Feb 6, 2014 at 9:35 AM, Abhinandan Prateek
abhinandan.prat...@citrix.commailto:abhinandan.prat...@citrix.com wrote:
+1, The listVM call is one of the most resource intensive call.
+Any
step
to optimise it are welcome.
On 06/02/14 2:01 pm, Koushik Das
koushik@citrix.commailto:koushik@citrix.com wrote:
Currently list VM can only be called using a single VM ID. So if
there is a need to query a set of VMs using ID then either
multiple list VM calls need to be made or all VMs needs to be
fetched and then do a client side filtering. Both approaches are
sub-optimal - the former results in multiple queries to database
and the latter will be an overkill if you need a small subset
from a very large number of VMs.
The proposal is to have an additional parameter to specify a list
of VM IDs for which the data needs to be fetched. Using this the
required VMs can be queried in an efficient manner. With the new
parameter the syntax would look like
http://localhost:8096/api?command=listVirtualMachineslistAll=true
i
ds=
edd
ac053-9b12-4d2e-acb7-233de2e98112,009966fc-4d7b-4f84-8609-
254979ba01
34
The new 'ids' parameter will be mutually exclusive with the
existing 'id'
parameter.
Let me know if there are any concerns/comments.
Thanks,
Koushik
--
Daan
RE: DB upgrade 4.2.1 - 4.3 fails
On 07.02.2014 19:01, Animesh Chaturvedi wrote: Nux you had a similar issue for previous RC https://issues.apache.org/jira/browse/CLOUDSTACK-5902 which was resolved as not a problem I know, but this time it's persistent. I think it may be related to the installation of the new system template. As specified in the bug, at what time must the new template be installed? I did it while still in running 4.2.1 and no luck, did it after, still no luck. -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro
Re: [PROPOSAL] load balancing configuration for VM nic secondary ips
+1. Looks fine to me. I think there are should be some UI changes, allow to choose multiple IPs if VM has more than one IP in this network. --Sheng On Fri, Feb 7, 2014 at 9:55 AM, Chiradeep Vittal chiradeep.vit...@citrix.com wrote: +1. Any UI changes? On 2/6/14 5:11 AM, Jayapal Reddy Uradi jayapalreddy.ur...@citrix.com wrote: Hi, Multiple ips per nic feature allows user to acquire more ip address on vm nic. On these ip addresses user able to configure PF and static NAT. With this feature user can also configure the load balancing rules for vm secondary ips also. Please review the below and give your comments. https://cwiki.apache.org/confluence/display/CLOUDSTACK/Configuring+load+ba lancing+rules+for+VM+nic+secondary+ips Ticket id: CLOUDSTACK-2692 Thanks, Jayapal
Re: commit 6523c068695d0431070060667c222eb40d54b14d breaks network removal
sure, will try to find a spot asap. and write unit tests to simulate
those two situations.
On Fri, Feb 7, 2014 at 7:20 PM, Alena Prokharchyk
alena.prokharc...@citrix.com wrote:
Daan,
Here is how it should look:
//1) Make all the checks that used to exist in original code + if DHCP
service is enabled on the network
if (vm.getType() == Type.User network.getTrafficType() ==
TrafficType.Guest isLastNicInSubnet(nic) network.getGuestType() ==
GuestType.Shared
_networkModel.areServicesSupportedInNetwork(network.getId(),Service.Dhcp))
{
//2) Now get the DHCP provider, and do the rest of the checks
DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
if (dhcpServiceProvider != null
isDhcpAccrossMultipleSubnetsSupported(dhcpServiceProvider)) {
removeDhcpServiceInSubnet(nic);
}
}
Could you please test it for 2 Shared networks - one with DHCP service,
and one w/o?
Thank you!
Alena.
On 2/7/14, 10:04 AM, Daan Hoogland daan.hoogl...@gmail.com wrote:
H Alena,
I am just trying to fix an old contribution that I applied as it
seemed not to harm in a basic test. revert didn't work so I am looking
for a quick remedy. The original patch does it for shared only. I
don't care either way. Lets do the best thing.
the code now
if (vm.getType() == Type.User
_networkModel.areServicesSupportedInNetwork(network.getId(),
Service.Dhcp)
network.getTrafficType() == TrafficType.Guest
network.getGuestType() == GuestType.Shared) {
// remove the dhcpservice ip if this is the last nic in
subnet.
DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
if (dhcpServiceProvider != null
isDhcpAccrossMultipleSubnetsSupported(dhcpServiceProvider)
isLastNicInSubnet(nic)) {
removeDhcpServiceInSubnet(nic);
}
}
What do you sugest?
if (vm.getType() == Type.User
_networkModel.areServicesSupportedInNetwork(network.getId(),
Service.Dhcp)) {
// remove the dhcpservice ip if this is the last nic in
subnet.
DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
if (dhcpServiceProvider != null
isDhcpAccrossMultipleSubnetsSupported(dhcpServiceProvider)
isLastNicInSubnet(nic)
network.getTrafficType() == TrafficType.Guest
network.getGuestType() == GuestType.Shared) {
removeDhcpServiceInSubnet(nic);
}
}
???
On Fri, Feb 7, 2014 at 6:56 PM, Alena Prokharchyk
alena.prokharc...@citrix.com wrote:
Daan,
1) What is the reason you execute this code snippet just for Shared
networks?
2) As I suggested in my prev email, before retrieving Dhcpprovider, you
should check if dhcp service is enabled on the network. Use method
areServicesSupportedInNetwork
From NetworkModel to check that.
-Alena.
On 2/6/14, 10:04 PM, Daan Hoogland daan.hoogl...@gmail.com wrote:
Alena,
The revert didn't apply. Would the folowing do the trick?
if (vm.getType() == Type.User
network.getTrafficType() == TrafficType.Guest
network.getGuestType() == GuestType.Shared) {
// remove the dhcpservice ip if this is the last nic in
subnet.
DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
if (dhcpServiceProvider != null
isDhcpAccrossMultipleSubnetsSupported(dhcpServiceProvider)
isLastNicInSubnet(nic)) {
removeDhcpServiceInSubnet(nic);
}
}
On Fri, Feb 7, 2014 at 6:55 AM, Daan Hoogland daan.hoogl...@gmail.com
wrote:
second thought,
Soheils mail bounces and the commit does not refer a ticket from jira.
I am going to revert. I should have been more vigilant. sorry.
On Fri, Feb 7, 2014 at 6:49 AM, Daan Hoogland
daan.hoogl...@gmail.com
wrote:
will do Alena,
thanks for the headsup
On Thu, Feb 6, 2014 at 10:42 PM, Alena Prokharchyk
alena.prokharc...@citrix.com wrote:
Soheil/Daan,
The commit in the subject breaks network System vms destroy (VR,
SSVM,
CPVM), resulting in the network removal failures. Following line
replacement
causes the failure:
- if (vm.getType() == Type.User
isDhcpAccrossMultipleSubnetsSupported(network)
isLastNicInSubnet(nic)
network.getTrafficType() == TrafficType.Guest
With
+DhcpServiceProvider dhcpServiceProvider =
getDhcpServiceProvider(network);
When you try to call getDhcpServiceProvider(network), it throws an
exception
because DHCP service is not enabled in Public/Control networks of
system vms
nics. So system vm always fails to expunge.
Could you please fix it by checking if DHCP service is enabled on
the
network, before getting the DHCP service provider?
Thanks,
Alena.
--
Daan
--
Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits
Hey Daan, I guess you are talking about your fix submitted in master [1]. You might want to see Hari's latest patch [2] as well. It solves the following two things which I guess your fix misses. 1. It sets the vcpu max to a hardcoded value (say 16) only when dynamic scaling is enabled. Do note that by default dynamic scaling is disabled. This is an important fix bcz only those who need dynamic scaling get impacted. 2. It has made the vcpu max configurable at cluster level which makes it flexible for the admin to customize it depending on the load in his/her cluster.(I guess even you guys mentioned keeping it configurable) In case there are issues with keeping it at 16, there is a flexibility to change it to a lower value during runtime. Let me know if you have any concerns. [1] https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blobdiff;f=plugi ns/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase .java;h=cf5c6d6c7623e682e6c5bd66d829351b2bf6ad49;hp=200a72ff219d5214ba3ebfc 2c198517e849e03a9;hb=0839fbc;hpb=b3829e54d6b7af426f797ffb9fa54b4cd2abffc0 [2] https://reviews.apache.org/r/17747/ Thanks, -Nitin On 06/02/14 10:10 PM, Daan Hoogland daan.hoogl...@gmail.com wrote: Animesh, I put in a patch that makes it double the number of assigned vcpu or 16 whichever is smaller. it is on 4.3-forward On Fri, Feb 7, 2014 at 6:28 AM, Animesh Chaturvedi animesh.chaturv...@citrix.com wrote: -Original Message- From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] Sent: Thursday, February 06, 2014 6:33 AM To: Harikrishna Patnala Cc: Nitin Mehta; cloudstack Subject: Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits we have hosts with 80 vms. 80*16 160 , which is spedcified in the xenserver docs Joris came up with. That last part is not important to me but I am still worried about the size of the statistics post by the members to the pool- master. If we can make sure we don't cross this boundary I am fine with not making it optional. So to stress my point: even with a documented limit of 16 per vm there is also a limit of 160 per host. And the real limit is neither as we can instantiate vms with 32 vcpu (even on 6.0.2 i think Joris?) the actual problem is in the internal xapi traffic. [Animesh] xapi traffic issues need to be addressed by xen and outside of cloudstack. From xen doc the vCPU per host is 4000 not 160 (which is logical processor/host). For 4.3 does it make sense to keep the value lower to like 8 to reduce chances of overloading xapi. For 4.4 this can be reworked to a configurable or computed value bases on different limits On Thu, Feb 6, 2014 at 1:47 PM, Harikrishna Patnala harikrishna.patn...@citrix.com wrote: This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17747/ On February 6th, 2014, 8:41 a.m. UTC, *daan Hoogland* wrote: Harikrishna, I would like to see the intermediate option of scale up to double the amount as well. Did you revert it? Is there a problem with this approach? Otherwise your submission is fine, of course. Hi Daan, I did not revert any changes. We can put an option to scale upto double but why it is required if vm can go till 16 (if at all 16 is the correct limit). - Harikrishna On February 5th, 2014, 5:19 p.m. UTC, Harikrishna Patnala wrote: Review request for cloudstack and Nitin Mehta. By Harikrishna Patnala. *Updated Feb. 5, 2014, 5:19 p.m.* *Bugs: * CLOUDSTACK-6023https://issues.apache.org/jira/browse/CLOUDSTACK- 6023 *Repository: * cloudstack-git Description CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits VCPUs-max value is changed to 16 and only when dynamic scaling is enabled. Diffs - plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixReso urceBase.java (bf9b068) View Diff https://reviews.apache.org/r/17747/diff/ -- Daan -- Daan
RE: DB upgrade 4.2.1 - 4.3 fails
On 07.02.2014 19:47, Animesh Chaturvedi wrote: Did you review Alena's response? Of course, check the issue https://issues.apache.org/jira/browse/CLOUDSTACK-6061 -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro
RE: DB upgrade 4.2.1 - 4.3 fails
Alena can you review the comment from Nux in bug -Original Message- From: Nux! [mailto:n...@li.nux.ro] Sent: Friday, February 07, 2014 11:52 AM To: dev@cloudstack.apache.org Subject: RE: DB upgrade 4.2.1 - 4.3 fails On 07.02.2014 19:47, Animesh Chaturvedi wrote: Did you review Alena's response? Of course, check the issue https://issues.apache.org/jira/browse/CLOUDSTACK-6061 -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro
RE: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits
Can I have the patch applied in 4.3-forward? -Original Message- From: Nitin Mehta [mailto:nitin.me...@citrix.com] Sent: Friday, February 07, 2014 11:33 AM To: dev@cloudstack.apache.org Cc: Harikrishna Patnala; Anthony Xu Subject: Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits Hey Daan, I guess you are talking about your fix submitted in master [1]. You might want to see Hari's latest patch [2] as well. It solves the following two things which I guess your fix misses. 1. It sets the vcpu max to a hardcoded value (say 16) only when dynamic scaling is enabled. Do note that by default dynamic scaling is disabled. This is an important fix bcz only those who need dynamic scaling get impacted. 2. It has made the vcpu max configurable at cluster level which makes it flexible for the admin to customize it depending on the load in his/her cluster.(I guess even you guys mentioned keeping it configurable) In case there are issues with keeping it at 16, there is a flexibility to change it to a lower value during runtime. Let me know if you have any concerns. [1] https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blobdiff;f=plugi ns/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceB ase .java;h=cf5c6d6c7623e682e6c5bd66d829351b2bf6ad49;hp=200a72ff219d5214 ba3ebfc 2c198517e849e03a9;hb=0839fbc;hpb=b3829e54d6b7af426f797ffb9fa54b4cd2 abffc0 [2] https://reviews.apache.org/r/17747/ Thanks, -Nitin On 06/02/14 10:10 PM, Daan Hoogland daan.hoogl...@gmail.com wrote: Animesh, I put in a patch that makes it double the number of assigned vcpu or 16 whichever is smaller. it is on 4.3-forward On Fri, Feb 7, 2014 at 6:28 AM, Animesh Chaturvedi animesh.chaturv...@citrix.com wrote: -Original Message- From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] Sent: Thursday, February 06, 2014 6:33 AM To: Harikrishna Patnala Cc: Nitin Mehta; cloudstack Subject: Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits we have hosts with 80 vms. 80*16 160 , which is spedcified in the xenserver docs Joris came up with. That last part is not important to me but I am still worried about the size of the statistics post by the members to the pool- master. If we can make sure we don't cross this boundary I am fine with not making it optional. So to stress my point: even with a documented limit of 16 per vm there is also a limit of 160 per host. And the real limit is neither as we can instantiate vms with 32 vcpu (even on 6.0.2 i think Joris?) the actual problem is in the internal xapi traffic. [Animesh] xapi traffic issues need to be addressed by xen and outside of cloudstack. From xen doc the vCPU per host is 4000 not 160 (which is logical processor/host). For 4.3 does it make sense to keep the value lower to like 8 to reduce chances of overloading xapi. For 4.4 this can be reworked to a configurable or computed value bases on different limits On Thu, Feb 6, 2014 at 1:47 PM, Harikrishna Patnala harikrishna.patn...@citrix.com wrote: This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17747/ On February 6th, 2014, 8:41 a.m. UTC, *daan Hoogland* wrote: Harikrishna, I would like to see the intermediate option of scale up to double the amount as well. Did you revert it? Is there a problem with this approach? Otherwise your submission is fine, of course. Hi Daan, I did not revert any changes. We can put an option to scale upto double but why it is required if vm can go till 16 (if at all 16 is the correct limit). - Harikrishna On February 5th, 2014, 5:19 p.m. UTC, Harikrishna Patnala wrote: Review request for cloudstack and Nitin Mehta. By Harikrishna Patnala. *Updated Feb. 5, 2014, 5:19 p.m.* *Bugs: * CLOUDSTACK- 6023https://issues.apache.org/jira/browse/CLOUDSTACK- 6023 *Repository: * cloudstack-git Description CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits VCPUs-max value is changed to 16 and only when dynamic scaling is enabled. Diffs - plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixR eso urceBase.java (bf9b068) View Diff https://reviews.apache.org/r/17747/diff/ -- Daan -- Daan
RE: DB upgrade 4.2.1 - 4.3 fails
Did you review Alena's response? -Original Message- From: Nux! [mailto:n...@li.nux.ro] Sent: Friday, February 07, 2014 11:28 AM To: dev@cloudstack.apache.org Subject: RE: DB upgrade 4.2.1 - 4.3 fails On 07.02.2014 19:01, Animesh Chaturvedi wrote: Nux you had a similar issue for previous RC https://issues.apache.org/jira/browse/CLOUDSTACK-5902 which was resolved as not a problem I know, but this time it's persistent. I think it may be related to the installation of the new system template. As specified in the bug, at what time must the new template be installed? I did it while still in running 4.2.1 and no luck, did it after, still no luck. -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro
Re: [ANNOUNCE] New PMC Member: Mike Tutkowski
Thanks, everyone! On Tue, Feb 4, 2014 at 9:06 PM, Santhosh Edukulla santhosh.eduku...@citrix.com wrote: Congrats Mike!! Santhosh From: Ahmad Emneina [aemne...@gmail.com] Sent: Tuesday, February 04, 2014 10:45 PM To: dev@cloudstack.apache.org Subject: Re: [ANNOUNCE] New PMC Member: Mike Tutkowski whoa, great news. Congrats Mike! On Tue, Feb 4, 2014 at 1:33 PM, Animesh Chaturvedi animesh.chaturv...@citrix.com wrote: Congrats Mike -Original Message- From: Chip Childers [mailto:chipchild...@apache.org] Sent: Tuesday, February 04, 2014 8:21 AM To: dev@cloudstack.apache.org Subject: [ANNOUNCE] New PMC Member: Mike Tutkowski The Project Management Committee (PMC) for Apache CloudStack has asked Mike Tutkowski to join the PMC and we are pleased to announce that he has accepted. Join me in congratulating Mike! -The CloudStack PMC -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloudhttp://solidfire.com/solution/overview/?video=play *(tm)*
RE: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits
-Original Message- From: Nitin Mehta [mailto:nitin.me...@citrix.com] Sent: Friday, February 07, 2014 11:33 AM To: dev@cloudstack.apache.org Cc: Harikrishna Patnala; Anthony Xu Subject: Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits Hey Daan, I guess you are talking about your fix submitted in master [1]. You might want to see Hari's latest patch [2] as well. It solves the following two things which I guess your fix misses. 1. It sets the vcpu max to a hardcoded value (say 16) only when dynamic scaling is enabled. Do note that by default dynamic scaling is disabled. This is an important fix bcz only those who need dynamic scaling get impacted. 2. It has made the vcpu max configurable at cluster level which makes it flexible for the admin to customize it depending on the load in his/her cluster.(I guess even you guys mentioned keeping it configurable) In case there are issues with keeping it at 16, there is a flexibility to change it to a lower value during runtime. [Animesh] Daan making this configurable allows much more flexibility so I think we do not need the hard coded twice the assignedvcpu Let me know if you have any concerns. [1] https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blobdiff;f=plugi ns/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceB ase .java;h=cf5c6d6c7623e682e6c5bd66d829351b2bf6ad49;hp=200a72ff219d5214 ba3ebfc 2c198517e849e03a9;hb=0839fbc;hpb=b3829e54d6b7af426f797ffb9fa54b4cd2 abffc0 [2] https://reviews.apache.org/r/17747/ Thanks, -Nitin On 06/02/14 10:10 PM, Daan Hoogland daan.hoogl...@gmail.com wrote: Animesh, I put in a patch that makes it double the number of assigned vcpu or 16 whichever is smaller. it is on 4.3-forward On Fri, Feb 7, 2014 at 6:28 AM, Animesh Chaturvedi animesh.chaturv...@citrix.com wrote: -Original Message- From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] Sent: Thursday, February 06, 2014 6:33 AM To: Harikrishna Patnala Cc: Nitin Mehta; cloudstack Subject: Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits we have hosts with 80 vms. 80*16 160 , which is spedcified in the xenserver docs Joris came up with. That last part is not important to me but I am still worried about the size of the statistics post by the members to the pool- master. If we can make sure we don't cross this boundary I am fine with not making it optional. So to stress my point: even with a documented limit of 16 per vm there is also a limit of 160 per host. And the real limit is neither as we can instantiate vms with 32 vcpu (even on 6.0.2 i think Joris?) the actual problem is in the internal xapi traffic. [Animesh] xapi traffic issues need to be addressed by xen and outside of cloudstack. From xen doc the vCPU per host is 4000 not 160 (which is logical processor/host). For 4.3 does it make sense to keep the value lower to like 8 to reduce chances of overloading xapi. For 4.4 this can be reworked to a configurable or computed value bases on different limits On Thu, Feb 6, 2014 at 1:47 PM, Harikrishna Patnala harikrishna.patn...@citrix.com wrote: This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17747/ On February 6th, 2014, 8:41 a.m. UTC, *daan Hoogland* wrote: Harikrishna, I would like to see the intermediate option of scale up to double the amount as well. Did you revert it? Is there a problem with this approach? Otherwise your submission is fine, of course. Hi Daan, I did not revert any changes. We can put an option to scale upto double but why it is required if vm can go till 16 (if at all 16 is the correct limit). - Harikrishna On February 5th, 2014, 5:19 p.m. UTC, Harikrishna Patnala wrote: Review request for cloudstack and Nitin Mehta. By Harikrishna Patnala. *Updated Feb. 5, 2014, 5:19 p.m.* *Bugs: * CLOUDSTACK- 6023https://issues.apache.org/jira/browse/CLOUDSTACK- 6023 *Repository: * cloudstack-git Description CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits VCPUs-max value is changed to 16 and only when dynamic scaling is enabled. Diffs - plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixR eso urceBase.java (bf9b068) View Diff https://reviews.apache.org/r/17747/diff/ -- Daan -- Daan
Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits
On Feb. 7, 2014, 3:57 p.m., Nitin Mehta wrote: plugins/hypervisors/xen/src/com/cloud/hypervisor/XenServerGuru.java, line 70 https://reviews.apache.org/r/17747/diff/2/?file=480018#file480018line70 Will the upgrade path get this config ?? Yes Nitin, after upgrade or fresh 4.3 setup upon start of management server ConfigDepot parses all managers for the configuration parameters and puts in DB if it not there or Updates if any new changes are there. - Harikrishna --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17747/#review33928 --- On Feb. 7, 2014, 11:19 a.m., Harikrishna Patnala wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17747/ --- (Updated Feb. 7, 2014, 11:19 a.m.) Review request for cloudstack, Abhinandan Prateek, Kishan Kavala, and Nitin Mehta. Bugs: CLOUDSTACK-6023 https://issues.apache.org/jira/browse/CLOUDSTACK-6023 Repository: cloudstack-git Description --- CLOUDSTACK-6023: Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits Changed the VCPU max limit to 16 and provided a cluster level configuration parameter for this max limit named xen.vm.vcpu.max Diffs - api/src/com/cloud/agent/api/to/VirtualMachineTO.java e6240ff plugins/hypervisors/xen/src/com/cloud/hypervisor/XenServerGuru.java 28bd724 plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java 48ccf70 Diff: https://reviews.apache.org/r/17747/diff/ Testing --- Thanks, Harikrishna Patnala
Re: DB upgrade 4.2.1 - 4.3 fails
See my reply to the bug. The new system vm template should be registered through CS API registerTemplate (and named like Install Guide requests), not by running template install script. Thanks! Alena. On 2/7/14, 11:51 AM, Nux! n...@li.nux.ro wrote: On 07.02.2014 19:47, Animesh Chaturvedi wrote: Did you review Alena's response? Of course, check the issue https://issues.apache.org/jira/browse/CLOUDSTACK-6061 -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro
Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17747/#review33959 --- Commit 95e41fdf0da50c165a9317847058fce4efeddbcf in branch refs/heads/4.3-forward from Harikrishna Patnala [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=95e41fd ] CLOUDSTACK-6023: Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits Changed the VCPU max limit to 16 and provided a cluster level configuration parameter for this max limit named xen.vm.vcpu.max - ASF Subversion and Git Services On Feb. 7, 2014, 11:19 a.m., Harikrishna Patnala wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17747/ --- (Updated Feb. 7, 2014, 11:19 a.m.) Review request for cloudstack, Abhinandan Prateek, Kishan Kavala, and Nitin Mehta. Bugs: CLOUDSTACK-6023 https://issues.apache.org/jira/browse/CLOUDSTACK-6023 Repository: cloudstack-git Description --- CLOUDSTACK-6023: Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits Changed the VCPU max limit to 16 and provided a cluster level configuration parameter for this max limit named xen.vm.vcpu.max Diffs - api/src/com/cloud/agent/api/to/VirtualMachineTO.java e6240ff plugins/hypervisors/xen/src/com/cloud/hypervisor/XenServerGuru.java 28bd724 plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java 48ccf70 Diff: https://reviews.apache.org/r/17747/diff/ Testing --- Thanks, Harikrishna Patnala
Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits
Animesh - done. Please cherry pick to 4.3 Daan - for pushing this to master I will have to revert your fix in master. On 07/02/14 11:48 AM, Animesh Chaturvedi animesh.chaturv...@citrix.com wrote: Can I have the patch applied in 4.3-forward? -Original Message- From: Nitin Mehta [mailto:nitin.me...@citrix.com] Sent: Friday, February 07, 2014 11:33 AM To: dev@cloudstack.apache.org Cc: Harikrishna Patnala; Anthony Xu Subject: Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits Hey Daan, I guess you are talking about your fix submitted in master [1]. You might want to see Hari's latest patch [2] as well. It solves the following two things which I guess your fix misses. 1. It sets the vcpu max to a hardcoded value (say 16) only when dynamic scaling is enabled. Do note that by default dynamic scaling is disabled. This is an important fix bcz only those who need dynamic scaling get impacted. 2. It has made the vcpu max configurable at cluster level which makes it flexible for the admin to customize it depending on the load in his/her cluster.(I guess even you guys mentioned keeping it configurable) In case there are issues with keeping it at 16, there is a flexibility to change it to a lower value during runtime. Let me know if you have any concerns. [1] https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blobdiff;f=plu gi ns/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceB ase .java;h=cf5c6d6c7623e682e6c5bd66d829351b2bf6ad49;hp=200a72ff219d5214 ba3ebfc 2c198517e849e03a9;hb=0839fbc;hpb=b3829e54d6b7af426f797ffb9fa54b4cd2 abffc0 [2] https://reviews.apache.org/r/17747/ Thanks, -Nitin On 06/02/14 10:10 PM, Daan Hoogland daan.hoogl...@gmail.com wrote: Animesh, I put in a patch that makes it double the number of assigned vcpu or 16 whichever is smaller. it is on 4.3-forward On Fri, Feb 7, 2014 at 6:28 AM, Animesh Chaturvedi animesh.chaturv...@citrix.com wrote: -Original Message- From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] Sent: Thursday, February 06, 2014 6:33 AM To: Harikrishna Patnala Cc: Nitin Mehta; cloudstack Subject: Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits we have hosts with 80 vms. 80*16 160 , which is spedcified in the xenserver docs Joris came up with. That last part is not important to me but I am still worried about the size of the statistics post by the members to the pool- master. If we can make sure we don't cross this boundary I am fine with not making it optional. So to stress my point: even with a documented limit of 16 per vm there is also a limit of 160 per host. And the real limit is neither as we can instantiate vms with 32 vcpu (even on 6.0.2 i think Joris?) the actual problem is in the internal xapi traffic. [Animesh] xapi traffic issues need to be addressed by xen and outside of cloudstack. From xen doc the vCPU per host is 4000 not 160 (which is logical processor/host). For 4.3 does it make sense to keep the value lower to like 8 to reduce chances of overloading xapi. For 4.4 this can be reworked to a configurable or computed value bases on different limits On Thu, Feb 6, 2014 at 1:47 PM, Harikrishna Patnala harikrishna.patn...@citrix.com wrote: This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17747/ On February 6th, 2014, 8:41 a.m. UTC, *daan Hoogland* wrote: Harikrishna, I would like to see the intermediate option of scale up to double the amount as well. Did you revert it? Is there a problem with this approach? Otherwise your submission is fine, of course. Hi Daan, I did not revert any changes. We can put an option to scale upto double but why it is required if vm can go till 16 (if at all 16 is the correct limit). - Harikrishna On February 5th, 2014, 5:19 p.m. UTC, Harikrishna Patnala wrote: Review request for cloudstack and Nitin Mehta. By Harikrishna Patnala. *Updated Feb. 5, 2014, 5:19 p.m.* *Bugs: * CLOUDSTACK- 6023https://issues.apache.org/jira/browse/CLOUDSTACK- 6023 *Repository: * cloudstack-git Description CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits VCPUs-max value is changed to 16 and only when dynamic scaling is enabled. Diffs - plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixR eso urceBase.java (bf9b068) View Diff https://reviews.apache.org/r/17747/diff/ -- Daan -- Daan
RE: systemvm-4.3.0.iso
I have pulled it into next RC but for each fix going in 4.3-forward I want to see a matching bugId. -Original Message- From: Paul Angus [mailto:paul.an...@shapeblue.com] Sent: Friday, February 07, 2014 5:26 AM To: dev@cloudstack.apache.org Cc: Rajesh Battala; Animesh Chaturvedi Subject: RE: systemvm-4.3.0.iso Thanks Devdeep, @Animesh, can we make sure the patch gets into the next RC please. Many thanks Paul. Regards, Paul Angus Cloud Architect S: +44 20 3603 0540 | M: +447711418784 | T: @CloudyAngus paul.an...@shapeblue.com -Original Message- From: Devdeep Singh [mailto:devdeep.si...@citrix.com] Sent: 07 February 2014 12:13 To: dev@cloudstack.apache.org Cc: Rajesh Battala Subject: RE: systemvm-4.3.0.iso Hi Paul, Looks like you are using untagged vlan in your setup. There was a bug in agent code with respect to that. Rajesh made a fix for it in commit-id 7df148c762510f669b60edff9d8d690c800b1f95 few days back. This fix should address the problem. Regards, Devdeep -Original Message- From: Paul Angus [mailto:paul.an...@shapeblue.com] Sent: Friday, February 7, 2014 4:55 PM To: dev@cloudstack.apache.org Subject: RE: systemvm-4.3.0.iso Thanks for the info Devdeep, so yes the systemvm iso is getting copied to the local disk, but I'm getting the following error: ERROR HypervResource.HypervResourceController [6b05182a-efa1-4105- bc5c-0a49c55674f0] - com.cloud.agent.api.StartCommand fail on exceptionObject reference not set to an instance of an object. System.NullReferenceException: Object reference not set to an instance of an object. at HypervResource.WmiCallsV2.DeployVirtualMachine(Object jsonObj, String systemVmIso) at HypervResource.HypervResourceController.StartCommand(Object cmd) agent log: 2014-02-07 11:19:56,685 [28] DEBUG HypervResource.WmiCallsV2 [6b05182a- efa1-4105-bc5c-0a49c55674f0] - Going ahead with create VM s-27-VM, 1 vcpus, 512MB RAM 2014-02-07 11:19:56,716 [28] DEBUG HypervResource.WmiCallsV2 [6b05182a- efa1-4105-bc5c-0a49c55674f0] - Created VM s-27-VM 2014-02-07 11:19:57,075 [28] INFO HypervResource.WmiCallsV2 [6b05182a- efa1-4105-bc5c-0a49c55674f0] - VM with display name s-27-VM has GUID 9E6D3606-8210-4E84-8796-BEB7970785F4 2014-02-07 11:19:57,075 [28] DEBUG HypervResource.WmiCallsV2 [6b05182a- efa1-4105-bc5c-0a49c55674f0] - Resources for vm s-27-VM: 512 MB memory, 1 vcpus 2014-02-07 11:19:57,106 [28] DEBUG HypervResource.WmiCallsV2 [6b05182a- efa1-4105-bc5c-0a49c55674f0] - New controller type Microsoft:Hyper- V:Synthetic SCSI Controller WMI path is \\WIN- G23HSIAIU4O\root\virtualization\v2:Msvm_ResourceAllocationSettingData.I nstanceID=Microsoft:9E6D3606-8210-4E84-8796-BEB7970785F4\\57035109- ED28-4F65-8BCB-487CFC66F8E8\\0s 2014-02-07 11:19:57,106 [28] DEBUG HypervResource.WmiCallsV2 [6b05182a- efa1-4105-bc5c-0a49c55674f0] - Going to create s-27-VM with attached voluem ROOT-27 at \\10.0.1.27\hypervPri\stor\Primary\ROOT-27.vhd 2014-02-07 11:19:57,106 [28] DEBUG HypervResource.WmiCallsV2 [6b05182a- efa1-4105-bc5c-0a49c55674f0] - Create disk type Microsoft:Hyper-V:Synthetic Disk Drive (Named: ROOT-27), on vm s-27-VM , inserting disk\\10.0.1.27\hypervPri\stor\Primary\ROOT-27.vhd 2014-02-07 11:19:57,106 [28] DEBUG HypervResource.WmiCallsV2 [6b05182a- efa1-4105-bc5c-0a49c55674f0] - Creating DISK for VM s-27-VM (GUID 9E6D3606-8210-4E84-8796-BEB7970785F4) by attaching \\10.0.1.27\hypervPri\stor\Primary\ROOT-27.vhd 2014-02-07 11:19:57,403 [28] DEBUG HypervResource.WmiCallsV2 [6b05182a- efa1-4105-bc5c-0a49c55674f0] - Creating disk drive type Microsoft:Hyper- V:Synthetic Disk Drive, parent IDE controller is \\WIN- G23HSIAIU4O\root\virtualization\v2:Msvm_ResourceAllocationSettingData.I nstanceID=Microsoft:9E6D3606-8210-4E84-8796-BEB7970785F4\\83F8638B- 8DCA-4152-9EDA-2CA8B33039B4\\0 and address on controller is 0 2014-02-07 11:19:57,418 [28] DEBUG HypervResource.WmiCallsV2 [6b05182a- efa1-4105-bc5c-0a49c55674f0] - New disk drive type Microsoft:Hyper- V:Synthetic Disk Drive WMI path is \\WIN- G23HSIAIU4O\root\virtualization\v2:Msvm_ResourceAllocationSettingData.I nstanceID=Microsoft:9E6D3606-8210-4E84-8796-BEB7970785F4\\83F8638B- 8DCA-4152-9EDA-2CA8B33039B4\\0\\0\\Ds 2014-02-07 11:19:57,574 [28] INFO HypervResource.WmiCallsV2 [6b05182a- efa1-4105-bc5c-0a49c55674f0] - Created disk \\WIN- G23HSIAIU4O\root\virtualization\v2:Msvm_StorageAllocationSettingData.Ins tanceID=Microsoft:9E6D3606-8210-4E84-8796-BEB7970785F4\\83F8638B- 8DCA-4152-9EDA-2CA8B33039B4\\0\\0\\L for VM s-27-VM (GUID 9E6D3606- 8210-4E84-8796-BEB7970785F4), image \\10.0.1.27\hypervPri\stor\Primary\ROOT-27.vhd 2014-02-07 11:19:57,574 [28] ERROR HypervResource.HypervResourceController [6b05182a-efa1-4105-bc5c- 0a49c55674f0] - com.cloud.agent.api.StartCommand fail on exceptionObject reference not set to an instance of an object. System.NullReferenceException: Object reference not
RE: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits
Here is the commit id on 4.3-forward commit 95e41fdf0da50c165a9317847058fce4efeddbcf -Harikrishna From: Nitin Mehta Sent: Saturday, February 08, 2014 2:46 AM To: dev@cloudstack.apache.org Cc: Harikrishna Patnala; Anthony Xu Subject: Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits Animesh - done. Please cherry pick to 4.3 Daan - for pushing this to master I will have to revert your fix in master. On 07/02/14 11:48 AM, Animesh Chaturvedi animesh.chaturv...@citrix.com wrote: Can I have the patch applied in 4.3-forward? -Original Message- From: Nitin Mehta [mailto:nitin.me...@citrix.com] Sent: Friday, February 07, 2014 11:33 AM To: dev@cloudstack.apache.org Cc: Harikrishna Patnala; Anthony Xu Subject: Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits Hey Daan, I guess you are talking about your fix submitted in master [1]. You might want to see Hari's latest patch [2] as well. It solves the following two things which I guess your fix misses. 1. It sets the vcpu max to a hardcoded value (say 16) only when dynamic scaling is enabled. Do note that by default dynamic scaling is disabled. This is an important fix bcz only those who need dynamic scaling get impacted. 2. It has made the vcpu max configurable at cluster level which makes it flexible for the admin to customize it depending on the load in his/her cluster.(I guess even you guys mentioned keeping it configurable) In case there are issues with keeping it at 16, there is a flexibility to change it to a lower value during runtime. Let me know if you have any concerns. [1] https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blobdiff;f=plu gi ns/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceB ase .java;h=cf5c6d6c7623e682e6c5bd66d829351b2bf6ad49;hp=200a72ff219d5214 ba3ebfc 2c198517e849e03a9;hb=0839fbc;hpb=b3829e54d6b7af426f797ffb9fa54b4cd2 abffc0 [2] https://reviews.apache.org/r/17747/ Thanks, -Nitin On 06/02/14 10:10 PM, Daan Hoogland daan.hoogl...@gmail.com wrote: Animesh, I put in a patch that makes it double the number of assigned vcpu or 16 whichever is smaller. it is on 4.3-forward On Fri, Feb 7, 2014 at 6:28 AM, Animesh Chaturvedi animesh.chaturv...@citrix.com wrote: -Original Message- From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] Sent: Thursday, February 06, 2014 6:33 AM To: Harikrishna Patnala Cc: Nitin Mehta; cloudstack Subject: Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits we have hosts with 80 vms. 80*16 160 , which is spedcified in the xenserver docs Joris came up with. That last part is not important to me but I am still worried about the size of the statistics post by the members to the pool- master. If we can make sure we don't cross this boundary I am fine with not making it optional. So to stress my point: even with a documented limit of 16 per vm there is also a limit of 160 per host. And the real limit is neither as we can instantiate vms with 32 vcpu (even on 6.0.2 i think Joris?) the actual problem is in the internal xapi traffic. [Animesh] xapi traffic issues need to be addressed by xen and outside of cloudstack. From xen doc the vCPU per host is 4000 not 160 (which is logical processor/host). For 4.3 does it make sense to keep the value lower to like 8 to reduce chances of overloading xapi. For 4.4 this can be reworked to a configurable or computed value bases on different limits On Thu, Feb 6, 2014 at 1:47 PM, Harikrishna Patnala harikrishna.patn...@citrix.com wrote: This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17747/ On February 6th, 2014, 8:41 a.m. UTC, *daan Hoogland* wrote: Harikrishna, I would like to see the intermediate option of scale up to double the amount as well. Did you revert it? Is there a problem with this approach? Otherwise your submission is fine, of course. Hi Daan, I did not revert any changes. We can put an option to scale upto double but why it is required if vm can go till 16 (if at all 16 is the correct limit). - Harikrishna On February 5th, 2014, 5:19 p.m. UTC, Harikrishna Patnala wrote: Review request for cloudstack and Nitin Mehta. By Harikrishna Patnala. *Updated Feb. 5, 2014, 5:19 p.m.* *Bugs: * CLOUDSTACK- 6023https://issues.apache.org/jira/browse/CLOUDSTACK- 6023 *Repository: * cloudstack-git Description CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits VCPUs-max value is changed to 16 and only when dynamic
RE: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits
Done applied to 4.3 -Original Message- From: Harikrishna Patnala Sent: Friday, February 07, 2014 1:31 PM To: Nitin Mehta; dev@cloudstack.apache.org Cc: Anthony Xu; Animesh Chaturvedi Subject: RE: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits Here is the commit id on 4.3-forward commit 95e41fdf0da50c165a9317847058fce4efeddbcf -Harikrishna From: Nitin Mehta Sent: Saturday, February 08, 2014 2:46 AM To: dev@cloudstack.apache.org Cc: Harikrishna Patnala; Anthony Xu Subject: Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits Animesh - done. Please cherry pick to 4.3 Daan - for pushing this to master I will have to revert your fix in master. On 07/02/14 11:48 AM, Animesh Chaturvedi animesh.chaturv...@citrix.com wrote: Can I have the patch applied in 4.3-forward? -Original Message- From: Nitin Mehta [mailto:nitin.me...@citrix.com] Sent: Friday, February 07, 2014 11:33 AM To: dev@cloudstack.apache.org Cc: Harikrishna Patnala; Anthony Xu Subject: Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits Hey Daan, I guess you are talking about your fix submitted in master [1]. You might want to see Hari's latest patch [2] as well. It solves the following two things which I guess your fix misses. 1. It sets the vcpu max to a hardcoded value (say 16) only when dynamic scaling is enabled. Do note that by default dynamic scaling is disabled. This is an important fix bcz only those who need dynamic scaling get impacted. 2. It has made the vcpu max configurable at cluster level which makes it flexible for the admin to customize it depending on the load in his/her cluster.(I guess even you guys mentioned keeping it configurable) In case there are issues with keeping it at 16, there is a flexibility to change it to a lower value during runtime. Let me know if you have any concerns. [1] https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blobdiff;f= plu gi ns/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourc eB ase .java;h=cf5c6d6c7623e682e6c5bd66d829351b2bf6ad49;hp=200a72ff219d5 214 ba3ebfc 2c198517e849e03a9;hb=0839fbc;hpb=b3829e54d6b7af426f797ffb9fa54b4cd2 abffc0 [2] https://reviews.apache.org/r/17747/ Thanks, -Nitin On 06/02/14 10:10 PM, Daan Hoogland daan.hoogl...@gmail.com wrote: Animesh, I put in a patch that makes it double the number of assigned vcpu or 16 whichever is smaller. it is on 4.3-forward On Fri, Feb 7, 2014 at 6:28 AM, Animesh Chaturvedi animesh.chaturv...@citrix.com wrote: -Original Message- From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] Sent: Thursday, February 06, 2014 6:33 AM To: Harikrishna Patnala Cc: Nitin Mehta; cloudstack Subject: Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits we have hosts with 80 vms. 80*16 160 , which is spedcified in the xenserver docs Joris came up with. That last part is not important to me but I am still worried about the size of the statistics post by the members to the pool- master. If we can make sure we don't cross this boundary I am fine with not making it optional. So to stress my point: even with a documented limit of 16 per vm there is also a limit of 160 per host. And the real limit is neither as we can instantiate vms with 32 vcpu (even on 6.0.2 i think Joris?) the actual problem is in the internal xapi traffic. [Animesh] xapi traffic issues need to be addressed by xen and outside of cloudstack. From xen doc the vCPU per host is 4000 not 160 (which is logical processor/host). For 4.3 does it make sense to keep the value lower to like 8 to reduce chances of overloading xapi. For 4.4 this can be reworked to a configurable or computed value bases on different limits On Thu, Feb 6, 2014 at 1:47 PM, Harikrishna Patnala harikrishna.patn...@citrix.com wrote: This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17747/ On February 6th, 2014, 8:41 a.m. UTC, *daan Hoogland* wrote: Harikrishna, I would like to see the intermediate option of scale up to double the amount as well. Did you revert it? Is there a problem with this approach? Otherwise your submission is fine, of course. Hi Daan, I did not revert any changes. We can put an option to scale upto double but why it is required if vm can go till 16 (if at all 16 is the correct limit). - Harikrishna On February
Re: DB upgrade 4.2.1 - 4.3 fails
On 07.02.2014 20:55, Alena Prokharchyk wrote: See my reply to the bug. The new system vm template should be registered through CS API registerTemplate (and named like Install Guide requests), not by running template install script. I have corrected that, but the upgrade still fails. Issue updated with logs. -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro
Re: DB upgrade 4.2.1 - 4.3 fails
Nux, During the rollback, you just applied 4.2.1 db dump. That doesn¹t drop the tables that were inserted by partial 4.2.14.3 upgrade. Refer to my last comment to the bug for the rollback steps. Thanks, Alena. On 2/7/14, 1:36 PM, Nux! n...@li.nux.ro wrote: On 07.02.2014 20:55, Alena Prokharchyk wrote: See my reply to the bug. The new system vm template should be registered through CS API registerTemplate (and named like Install Guide requests), not by running template install script. I have corrected that, but the upgrade still fails. Issue updated with logs. -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro
[DOCS] update on move to rst
Hi, A quick update on the move to .rst format. We now have four repos for the docs: git://git.apache.org/cloudstack-docs-rn.git git://git.apache.org/cloudstack-docs-admin.git git://git.apache.org/cloudstack-docs-install.git git://git.apache.org/cloudstack-docs.git All four are mirrored on github: https://github.com/apache/cloudstack-docs https://github.com/apache/cloudstack-docs-install https://github.com/apache/cloudstack-docs-admin https://github.com/apache/cloudstack-docs-rn All four are built on Readthedocs.org from the github mirror http://apache-cloudstack-release-notes.readthedocs.org/en/latest/ http://cloudstack-administration.readthedocs.org/en/latest/ http://cloudstack-installation.readthedocs.org http://cloudstack.readthedocs.org Building from the github mirror allows us to edit directly via github. In each site you will see a 'edit on github' button in the top left, if you click on it it will fork the site on your github account and you will be able to submit pull request. That's the main reason of doing this and should help everyone with contributing to the docs. The installation guide is almost cleaned-up. That means that the conversion to .rst is complete. But no edits or docs correction have been made. The administration guide is under going clean up The release notes have had no clean up, I just did the conversion from the 4.3 branch The cloudstack.rtd site contains the midonet/midokura and other smaller guides, it also has some new guides: ansible, trouble shooting, autoscaling without netscaler. David and I are going to setup the CNAMES in the coming days and finish the clean up of the formatting on all guides. If some of you want to help with the clean-up over the week end, go for it and start sending pull requests. Any help appreciated, -Sebastien
Re: DB upgrade 4.2.1 - 4.3 fails
On 07.02.2014 21:50, Alena Prokharchyk wrote: Nux, During the rollback, you just applied 4.2.1 db dump. That doesn¹t drop the tables that were inserted by partial 4.2.14.3 upgrade. Refer to my last comment to the bug for the rollback steps. Yes, that was it! On to more testing now, thanks! -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro
Re: Review Request 17233: Fix for Issue CLOUDSTACK-5875
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17233/#review33968 --- Commit c1af92fcbbc183873d6519f0660b68acfb33bd44 in branch refs/heads/4.3 from Santhosh Edukulla [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=c1af92f ] Added Fix for CLOUDSTACK-5875 Added fix for exception and listing. Mentioned details under bug. Post the fix, simulator works fine. Signed-off-by: Santhosh Edukulla santhosh.eduku...@citrix.com Signed-off-by: Koushik Das kous...@apache.org (cherry picked from commit d31fa09c7e5bb04fcc4594c0352d9b91171e9d1a) Signed-off-by: Animesh Chaturvedi anim...@apache.org - ASF Subversion and Git Services On Jan. 23, 2014, 10:27 a.m., Santhosh Edukulla wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17233/ --- (Updated Jan. 23, 2014, 10:27 a.m.) Review request for cloudstack and Koushik Das. Bugs: CLOUDSTACK-5875 https://issues.apache.org/jira/browse/CLOUDSTACK-5875 Repository: cloudstack-git Description --- Fixed the issue CLOUDSTACK-5875 The crash issue happens because of null value for CallContext.current() under discoverHostsFull, when called from thread for SystemVMHandler. Currently, there is no call context set for this thread. Currently, as part of MockAgentManagerImpl.java, SystemVMHandler implements Runnable, but when it calls discoverHosts, it does not have any call context set and it is crashing. Because of which agent was not coming up and template sync and resource discovery failed. Added a register and unregister of callcontext of system user for this thread. 2. There was a new column added by name state under vm_templates table. The value of this column for Active was checked for listing the templates if there was no removed attribute set for list templates api. This column was not present in 4.2. Added a column value Active for simulator schema. The value of this column was not Active and so listing failed. Testing: 1. Built the CS using the changes. Tested the column value, post deploydb and it was now set to Active. Deployed a DC and list template works. 2. Deploy DC worked fine with simulator . Created an instance using the template and it worked to start. Agent for systemvm was coming up. Note: 1. For listing templates, a hypervisor in the zone need to be present because listtemplates api is looking for it. Use the list templates post the deployDC and it works. 2. We need to set router.version.check to false under global properties of CS post run using simulator. Otherwise you may see errors related to router upgrade etc. This i believe is a known issue and nothing to do with simulator. 3. Post the changes, tested only with simulator. 3. Also, please change the component field. AS such it is not related to Marvin. Diffs - plugins/hypervisors/simulator/src/com/cloud/agent/manager/MockAgentManagerImpl.java 4071aeb setup/db/templates.simulator.sql 5f9b67d Diff: https://reviews.apache.org/r/17233/diff/ Testing --- Testing: 1. Built the CS 4.3-forward using the changes. Tested the column value change, post deploydb and it was now set to Active. Deployed a DC and list template works. 2. Deploy DC worked fine with simulator . Created an instance using the template and it worked to start. Agent for systemvm was coming up. Note: 1. For listing templates, a hypervisor in the zone need to be present because listtemplates api is looking for it. Use the list templates post the deployDC and it works. 2. We need to set router.version.check to false under global properties of CS post run using simulator. Otherwise you may see errors related to router upgrade etc. This i believe is a known issue and nothing to do with simulator. 3. Post the changes, tested only with simulator. Thanks, Santhosh Edukulla
Re: Review Request 17747: CLOUDSTACK-6023:Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17747/#review33967 --- Commit 01289b1647db2b08d42c883452aae6ca7dda in branch refs/heads/4.3 from Harikrishna Patnala [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=01289b1 ] CLOUDSTACK-6023: Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits Changed the VCPU max limit to 16 and provided a cluster level configuration parameter for this max limit named xen.vm.vcpu.max (cherry picked from commit 95e41fdf0da50c165a9317847058fce4efeddbcf) Signed-off-by: Animesh Chaturvedi anim...@apache.org - ASF Subversion and Git Services On Feb. 7, 2014, 11:19 a.m., Harikrishna Patnala wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17747/ --- (Updated Feb. 7, 2014, 11:19 a.m.) Review request for cloudstack, Abhinandan Prateek, Kishan Kavala, and Nitin Mehta. Bugs: CLOUDSTACK-6023 https://issues.apache.org/jira/browse/CLOUDSTACK-6023 Repository: cloudstack-git Description --- CLOUDSTACK-6023: Non windows instances are created on XenServer with a vcpu-max above supported xenserver limits Changed the VCPU max limit to 16 and provided a cluster level configuration parameter for this max limit named xen.vm.vcpu.max Diffs - api/src/com/cloud/agent/api/to/VirtualMachineTO.java e6240ff plugins/hypervisors/xen/src/com/cloud/hypervisor/XenServerGuru.java 28bd724 plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java 48ccf70 Diff: https://reviews.apache.org/r/17747/diff/ Testing --- Thanks, Harikrishna Patnala
Re: [PROPOSAL] List VM API enhancement
Yes. -min Sent from my iPhone On Feb 7, 2014, at 11:10 AM, Alena Prokharchyk alena.prokharc...@citrix.com wrote: We can just agree from now on to use the ³id for handling multiple ids. And of course, we can never delete the ³ID² parameter just to satisfy the old convention, as this is the most used parameter :) I can see that several existing commands - archive/deleteAlerts are using ApiConstants.IDs parameter. We can mark IDs as deprecated, so its no longer used by new commands. -Alena. On 2/7/14, 11:03 AM, Koushik Das koushik@citrix.com wrote: Good point Min. I also thought about it but looking at some of the existing APIs thought of keeping both. For e.g. in deploy VM api there is a parameter called 'networkids' which can take an array of network IDs. Note that the naming convention of ending in 's'. Now by this logic we should name the parameter 'ids' and remove the existing parameter 'id' which will be a breaking change. In case the existing 'id' parameter is used for multiple IDs that breaks the parameter naming convention. I am all in favour of using the existing 'id' parameter if there is no issues with breaking the naming convention. On 07-Feb-2014, at 11:25 PM, Min Chen min.c...@citrix.com wrote: Hi Koushik, I agree with the idea of supporting multiple IDs. But I may not like the idea of introducing another different query parameter ids for this purpose. Why cannot we just change current id parameter to take a list of values? This way, user will not need to use two different parameters for single or multiple cases. Maintaining two different parameters for similar purpose is error-prone. If you look at Amazon EC2 api, you will notice that they are also using the similar convention, id parameter can be one or more. Thanks -min On 2/6/14 3:24 AM, Koushik Das koushik@citrix.com wrote: Yes it will be like a findByIds() and the one id case is just a special case for this. On 06-Feb-2014, at 4:24 PM, Daan Hoogland daan.hoogl...@gmail.com wrote: looks nice, it will be backed by the current query for one id? or will you write a findByIds()? On Thu, Feb 6, 2014 at 9:35 AM, Abhinandan Prateek abhinandan.prat...@citrix.com wrote: +1, The listVM call is one of the most resource intensive call. Any step to optimise it are welcome. On 06/02/14 2:01 pm, Koushik Das koushik@citrix.com wrote: Currently list VM can only be called using a single VM ID. So if there is a need to query a set of VMs using ID then either multiple list VM calls need to be made or all VMs needs to be fetched and then do a client side filtering. Both approaches are sub-optimal - the former results in multiple queries to database and the latter will be an overkill if you need a small subset from a very large number of VMs. The proposal is to have an additional parameter to specify a list of VM IDs for which the data needs to be fetched. Using this the required VMs can be queried in an efficient manner. With the new parameter the syntax would look like http://localhost:8096/api?command=listVirtualMachineslistAll=trueid s= edd ac053-9b12-4d2e-acb7-233de2e98112,009966fc-4d7b-4f84-8609-254979ba013 4 The new 'ids' parameter will be mutually exclusive with the existing 'id' parameter. Let me know if there are any concerns/comments. Thanks, Koushik -- Daan
Re: [DISCUSS]null pointer returned from daoimpl
There is no good answer IMHO. The designer of the API chose this design.
Throwing a checked exception or returning Null is equivalent.
Throwing a runtime exception is probably wrong since there may be some
recovery possible.
We could annotate the method with @Nullable so that the compiler/IDE can
warn if the caller of the API forgets to check for Null.
On 2/7/14 2:30 AM, Daan Hoogland daan.hoogl...@gmail.com wrote:
LS,
in PrivateIpDaoImpl a null pointer is returned when no db object can be
found:
@Override
public PrivateIpVO allocateIpAddress(long dcId, long networkId,
String requestedIp) {
...
PrivateIpVO vo = lockOneRandomRow(sc, true);
if (vo == null) {
txn.rollback();
return null;
}
...
return vo;
}
I would expect it to throw a ClodException of some sort and would like
to change it to that but recognize that the null pointer could be of
significance in cases. Is there a policy on how dao's should return
failures?
In my opinion a null should never be returned by a dao, at most a vo
containing a null but this seldom makes sense.
--
Daan
RE: DB upgrade 4.2.1 - 4.3 fails
Ok so I am ready to roll with an RC. Thanks Nux, Alena -Original Message- From: Nux! [mailto:n...@li.nux.ro] Sent: Friday, February 07, 2014 2:09 PM To: dev@cloudstack.apache.org Subject: Re: DB upgrade 4.2.1 - 4.3 fails On 07.02.2014 21:50, Alena Prokharchyk wrote: Nux, During the rollback, you just applied 4.2.1 db dump. That doesn¹t drop the tables that were inserted by partial 4.2.14.3 upgrade. Refer to my last comment to the bug for the rollback steps. Yes, that was it! On to more testing now, thanks! -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro
RE: [DOCS] update on move to rst
Cool -Original Message- From: Sebastien Goasguen [mailto:run...@gmail.com] Sent: Friday, February 07, 2014 2:02 PM To: dev@cloudstack.apache.org Subject: [DOCS] update on move to rst Hi, A quick update on the move to .rst format. We now have four repos for the docs: git://git.apache.org/cloudstack-docs-rn.git git://git.apache.org/cloudstack-docs-admin.git git://git.apache.org/cloudstack-docs-install.git git://git.apache.org/cloudstack-docs.git All four are mirrored on github: https://github.com/apache/cloudstack-docs https://github.com/apache/cloudstack-docs-install https://github.com/apache/cloudstack-docs-admin https://github.com/apache/cloudstack-docs-rn All four are built on Readthedocs.org from the github mirror http://apache-cloudstack-release-notes.readthedocs.org/en/latest/ http://cloudstack-administration.readthedocs.org/en/latest/ http://cloudstack-installation.readthedocs.org http://cloudstack.readthedocs.org Building from the github mirror allows us to edit directly via github. In each site you will see a 'edit on github' button in the top left, if you click on it it will fork the site on your github account and you will be able to submit pull request. That's the main reason of doing this and should help everyone with contributing to the docs. The installation guide is almost cleaned-up. That means that the conversion to .rst is complete. But no edits or docs correction have been made. The administration guide is under going clean up The release notes have had no clean up, I just did the conversion from the 4.3 branch The cloudstack.rtd site contains the midonet/midokura and other smaller guides, it also has some new guides: ansible, trouble shooting, autoscaling without netscaler. David and I are going to setup the CNAMES in the coming days and finish the clean up of the formatting on all guides. If some of you want to help with the clean-up over the week end, go for it and start sending pull requests. Any help appreciated, -Sebastien
Re: [DOCS] update on move to rst
Visually so much nicer! On 2/7/14 2:58 PM, Animesh Chaturvedi animesh.chaturv...@citrix.com wrote: Cool -Original Message- From: Sebastien Goasguen [mailto:run...@gmail.com] Sent: Friday, February 07, 2014 2:02 PM To: dev@cloudstack.apache.org Subject: [DOCS] update on move to rst Hi, A quick update on the move to .rst format. We now have four repos for the docs: git://git.apache.org/cloudstack-docs-rn.git git://git.apache.org/cloudstack-docs-admin.git git://git.apache.org/cloudstack-docs-install.git git://git.apache.org/cloudstack-docs.git All four are mirrored on github: https://github.com/apache/cloudstack-docs https://github.com/apache/cloudstack-docs-install https://github.com/apache/cloudstack-docs-admin https://github.com/apache/cloudstack-docs-rn All four are built on Readthedocs.org from the github mirror http://apache-cloudstack-release-notes.readthedocs.org/en/latest/ http://cloudstack-administration.readthedocs.org/en/latest/ http://cloudstack-installation.readthedocs.org http://cloudstack.readthedocs.org Building from the github mirror allows us to edit directly via github. In each site you will see a 'edit on github' button in the top left, if you click on it it will fork the site on your github account and you will be able to submit pull request. That's the main reason of doing this and should help everyone with contributing to the docs. The installation guide is almost cleaned-up. That means that the conversion to .rst is complete. But no edits or docs correction have been made. The administration guide is under going clean up The release notes have had no clean up, I just did the conversion from the 4.3 branch The cloudstack.rtd site contains the midonet/midokura and other smaller guides, it also has some new guides: ansible, trouble shooting, autoscaling without netscaler. David and I are going to setup the CNAMES in the coming days and finish the clean up of the formatting on all guides. If some of you want to help with the clean-up over the week end, go for it and start sending pull requests. Any help appreciated, -Sebastien
Re: [PROPOSAL] Virtual Router aggregated execution
+1. * The guideline is not clear as to when a developer should use this executor. Why not use it all the time (even for a single command) * Are there any issues when there are multiple management servers involved? * Any threading concerns? That is, multiple threads are attempting to update the VR, some are using the aggregated approach, some are not. * What is the default value of aggregated period? * What if the caller dies before calling completeAggregatedExecution * what is the queue mechanism? LinkedBlockingQueue? * Any impact on the agent thread pool size? Does this use its own thread pool? * Can we also address the case of restoring state to a VR when restarting the VR outside of CloudStack. On 2/6/14 5:03 PM, Sheng Yang sh...@yasker.org wrote: Hi Devs, Here I'd like to introduce this improvement of VR. https://issues.apache.org/jira/browse/CLOUDSTACK-6047 https://cwiki.apache.org/confluence/display/CLOUDSTACK/Virtual+Router+aggr egated+command+execution In short, we would speed up VR's rebooting and re-creating, by aggregated execution the CloudStack configuration commands when booting up. Hopefully we can get it done in O(1) rather than O(n)(which is current state). And to prepare for this work, I've get a long expected code refactor done. Now one VirtualRoutingResources would take over all the VR execution commands(rather than every hypervisor's resource). From now on, you would only need to modify one place in order to update VR commands. I've put some details of VR aggregated execution in the FS. Comments are welcome! --Sheng
Re: Removing deploy\load options from marvinplugin
Phew. Thanks for the tldr On 2/7/14 3:53 AM, Prasanna Santhanam t...@apache.org wrote: tl;dr i think your point 4 clarifies your change. remove --load and only run tests unless --deploy is given. in the latter - do a deploy. On Fri, Feb 07, 2014 at 11:30:19AM +, Santhosh Edukulla wrote: I believe we are deviating with too many notes here. Lets put things in perspective. 1. Initial point was to understand and take inputs to have and work with marvinplugin using less options for running, minimize options we have currently and can we remove few and work with them? and i believe that's what you mentioned to have less options in earlier mail. 2. In the initial mail, it was mentioned that if there is a change, it will effect few areas like devcloud\simulator, provided if there is a change, starting this thread is to know a point of view and see the impact, that's what is to have clarified here. I see there is no impact there in other areas mentioned, that does not mean we are agreeing for a change. 3. Export\Import configuration from marvin\cloudstack is a separate issue for discussion, i believe you can include it in a separate thread for now and discuss there. People can have their say of having this facility or not. Regarding its a leak or not that's a separate discussion to have and how to design or implement again, that's nothing to do with options change we mentioned. This will keep the current discussions easier to follow. 4. deploy VS load, in the earlier mail, i didn't mentioned to remove deploy, i said only load option. Lets see what load option is doing currently, It does the below, which i believe can still be possible with one deploy option. Here, we are creating a client with configuration provided. This is happening even with load option and as well as inside of deploy option . I believe we can control this behavior with single deploy option. If deploy option is not provided, then it works as though load option else deploy option of currently. Please let me know where updating the global configuration is happening as part of current loadCfg option? def loadCfg(self): try: self.config = configGenerator.getSetupConfig(self.configFile) except: raise cloudstackException.InvalidParameterException( Failed to load config %s % self.configFile) ''' Retrieving Management Server Connection Details ''' mgtDetails = self.config.mgtSvr[0] ''' Retrieving Database Connection Details''' dbSvrDetails = self.config.dbSvr loggers = self.config.logger testClientLogFile = None self.testCaseLogFile = None self.testResultLogFile = None if loggers is not None and len(loggers) 0: for log in loggers: if log.name == TestClient: testClientLogFile = log.file elif log.name == TestCase: self.testCaseLogFile = log.file elif log.name == TestResult: self.testResultLogFile = log.file testClientLogger = None if testClientLogFile is not None: testClientLogger = logging.getLogger(testclient.testengine.run) fh = logging.FileHandler(testClientLogFile) fh.setFormatter(logging.Formatter( %(asctime)s - %(levelname)s - %(name)s\ - %(message)s) ) testClientLogger.addHandler(fh) testClientLogger.setLevel(logging.INFO) self.testClientLogger = testClientLogger self.testClient = \ cloudstackTestClient.\ cloudstackTestClient(mgtDetails, dbSvrDetails, logging=self.testClientLogger) logger=self.tcRunLogger) if mgtDetails.apiKey is None: mgtDetails.apiKey, mgtDetails.securityKey = self.registerApiKey()there run a deployDC with configuration provided and if not 5. Also, its better if know where we are upading the other global configuration you mentioned as part of load option? Here, its just creating the client based upon configuration provided. 6. why deploying cloudstack is part of nose tests now and where we mentioned it is and make it a 4 step process? We are anyways not doing it now as part of nosetests. We are adding one more addition of restart CS, which is totally not required as part of nosetets. Iam not sure adding a restart simplifies and makes it little more complex. 1. deploy cloudstack 2. deploydatacenter (done using nose earlier) 3. restart cloudstack 4. run tests (also done by nose earlier) 7. The reason for separation is to keep things simple. As a user, i can run below. The reason i mentioned to separate deploy out of nose tests is we are not doing anything as such to report a failure for bvt\regression etc for deployDC, we just exit
Re: Code quality, QA, etc
Hi, I used to work with gerrit in OS projects and I think the tool is great, the integration with jenkins is cool. One problem could be when jenkins infrastructure problems are frequent and developers start to ignore warnings from jenkins. With my particular project we were also frequently hit by gerrit outages. I do not know the reason since I did not operate the infrastructure, but having 1-2 outages per week was normal. This is the technical part and I am sure you can make a more reliable service. We also had Review-then-commit process, and in general I had a bad experience with the process. I do believe the code review is necessary in an open source project and it can improve quality, but at the same time the costs (in time and lost braincells) are very high and the existence of a process does not guarantee that the quality will improve. No process replaced thinking so far. Once I complained about having the 30th version of a patch that in my opinion was quite simple and then someone answered that he is already over the 40th review. It took several months to push something through the process. And those numbers just kept growing. We collected some of the top reasons with my team: - the review was not really a review, the reviewer only looked at the code in firefox. Never checked out, never ran the tests. - reviewer expectations were various even in the same language and module between reviewers, unfortunately this was not documented, so you had to use the try-and-fail process to learn individual reviewer preferences, it took quite a lot of time since the team was huge - one had to wait for review sometimes for several weeks. Meanwhile the patch got outdated and had to be rewritten, and then the whole process started over again. - Also, reviewers blocked at the first issue found in the patch. This was usually in the commit comment, they did not like it. So you change the commit comment and hope that next time the guy will read some actual code. Maybe he will block on something like he does not like your variable name. This is especially annoying when you send an urgent fix. - The typical reason for merging a patch was the release deadline. Just a few days before the deadline they merged everything. So we have spent several months and still only the developer tested the code. In my opinion a review tool is not enough to make the review process productive, you need good reviewers. Regards, Laszlo On Fri, Feb 7, 2014 at 4:50 AM, David Nalley da...@gnsa.us wrote: Hi folks, We continue to break things large and small in the codebase, and after a number of different conversations; I thought I'd bring that discussion here. First - coding quality is only one factor that the PMC considers when making someone a committer. Second - CloudStack is a huge codebase; has a ton of inter-related pieces, and unintended consequences are easy. We also have an pretty heady commit velocity - 20+ commits today alone. Some communities have Review-then-commit - which would slow us down, and presumably help us increase quality. However, I am not personally convinced that it will do so measurably because even the most experienced CloudStack developers occasionally break a build or worse. We could have an automated pipeline that verifies a number of different tests pass - before a patch/commit makes it into a mainline branch. That is difficult with our current tooling; but perhaps something worth considering. At FOSDEM, Hugo and I were discussing his experiences with Gerrit and OpenDaylight, and he thinks thats a viable option. I think it would certainly be a step in the right direction. Separately, Jake Farrell and I were discussing our git-related proposal for ApacheCon, and broached the subject of Gerrit. Jake is the current person bearing most of the load for git at the ASF, and he's also run Gerrit in other contexts. He points out a number of difficulties. (And I'd love for him to weigh in on this conversation, hence the CC) He wants to expand RB significantly, including pre-commit testing. So - thoughts, comments, flames? How do we improve code quality, stop needless breakage? Much of this is going to be cultural I think, and I personally think we struggle with that. Many folks have voiced an opinion about stopping continued commits when the build is broken; but we haven't been able to do that. --David -- EOF
Re: [PROPOSAL] Virtual Router aggregated execution
Thanks for the comment! On Fri, Feb 7, 2014 at 3:25 PM, Chiradeep Vittal chiradeep.vit...@citrix.com wrote: +1. * The guideline is not clear as to when a developer should use this executor. Why not use it all the time (even for a single command) The mechanism behind is NOT a producer-consumer model. It's just a queue(or a list) to delay the commands execution for now. Only the code would generated a large number of commands would need to use this, and basically that's the VR reboot/re-create at this time. * Are there any issues when there are multiple management servers involved? It would remain the same as before. One VR(host) would only operated by one mgmt server, the other one would reroute all the related commands to the mgmt server in charge. And as soon as aggregated execution started, it would block all the commands at least in VR level(also in the host queue if network.element.sequence.execution is true). * Any threading concerns? That is, multiple threads are attempting to update the VR, some are using the aggregated approach, some are not. It is not a multithread solution in my mind. One queue for one VR, and would be created only prepare func called. All the following commands, no matter from which thread, would goes to this queue(by hooking sendCommandsToRouter in VR), wait until complete func called(which is expected quite soon). * What is the default value of aggregated period? No default, commands would wait until complete func called. * What if the caller dies before calling completeAggregatedExecution As noted in the spec, exception handler should call abortAggregatedExecution(). As design for VR booting up period, one failure would means VR fail to boot up. And, before commands send to VR, all the exception should be handled by mgmt server. I don't think there are any factor we cannot control would impact the command generation. But nonetheless, abortAggregatedExecution() is provided to clean up the queue and fail the execution. * what is the queue mechanism? LinkedBlockingQueue? Since it's not async, any ordered queue should be fine. Probably I should call it ArrayList... * Any impact on the agent thread pool size? Does this use its own thread pool? No thread pool. * Can we also address the case of restoring state to a VR when restarting the VR outside of CloudStack. No. That's not in the scope. I was willing to take a more radical approach for this work, but practically I would want to solve the 80% problem this time(which is VR booting/upgrading time). --Sheng On 2/6/14 5:03 PM, Sheng Yang sh...@yasker.org wrote: Hi Devs, Here I'd like to introduce this improvement of VR. https://issues.apache.org/jira/browse/CLOUDSTACK-6047 https://cwiki.apache.org/confluence/display/CLOUDSTACK/Virtual+Router+aggr egated+command+execution In short, we would speed up VR's rebooting and re-creating, by aggregated execution the CloudStack configuration commands when booting up. Hopefully we can get it done in O(1) rather than O(n)(which is current state). And to prepare for this work, I've get a long expected code refactor done. Now one VirtualRoutingResources would take over all the VR execution commands(rather than every hypervisor's resource). From now on, you would only need to modify one place in order to update VR commands. I've put some details of VR aggregated execution in the FS. Comments are welcome! --Sheng
[VOTE] Apache CloudStack 4.3.0 (fourth round)
Hi All, I've created a 4.3.0 release, with the following artifacts up for a vote: Git Branch and Commit SH: https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/4.3 Commit: c1af92fcbbc183873d6519f0660b68acfb33bd44 List of changes: Release notes are still in progress in a different repo, so providing links here for now New Features in 4.3: https://issues.apache.org/jira/issues/?filter=12325248 Improvement in 4.3: https://issues.apache.org/jira/issues/?filter=12325249 Issues fixed in 4.3 https://issues.apache.org/jira/issues/?filter=12326161 Known Issues in 4.3: https://issues.apache.org/jira/issues/?filter=12326162 Source release (checksums and signatures are available at the same location): https://dist.apache.org/repos/dist/dev/cloudstack/4.3.0/ PGP release keys (signed using 94BE0D7C): https://dist.apache.org/repos/dist/release/cloudstack/KEYS Testing instructions are here: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Release+test+procedure Vote will be open for 3 working days and will close on Wednesday 2/12 5:00PM PST. For sanity in tallying the vote, can PMC members please be sure to indicate (binding) with their vote? [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why)
devs looking to get involved?
I occasionally have some minor improvement that I need to make to
whatever version of cloudstack we're currently running. This is almost
always behind what is currently being developed, and I don't always
feel like I have time to rebase/refactor it for master. I'm wondering
if there are any individuals out there who are looking for ways to
contribute that I can hand off these little projects to and mentor a
bit. You get an easy project to get your feet wet and I get the
improvements into master without using up time. :-) Maybe this sounds
like laziness on my part, but it seemed like a good idea.
I can post a patch that works for one version, and you can rework it
and test against master. Respond if you're interested, there may be
others who have easy work to farm off as well.
An example, I've made some improvements to the template downloader,
when the first 1M of the template is pulled we attempt to verify that
the image is actually what we think it is (qcow2 or vmdk or whatever)
by looking at the data. Up until now we just check file extension. It
also contains a new TemplateUtils class that has:
+public static boolean isCorrectExtension(String path, String ext) {
+if (path.toLowerCase().endsWith(ext)
+|| path.toLowerCase().endsWith(ext + .gz)
+|| path.toLowerCase().endsWith(ext + .bz2)
+|| path.toLowerCase().endsWith(ext + .zip)) {
+return true;
+}
+return false;
+}
Which can be used to clean up the likes of:
private void checkFormat(String format, String url) {
if((!url.toLowerCase().endsWith(vhd))(!url.toLowerCase().endsWith(vhd.zip))
(!url.toLowerCase().endsWith(vhd.bz2))(!url.toLowerCase().endsWith(vhd.gz))
(!url.toLowerCase().endsWith(qcow2))(!url.toLowerCase().endsWith(qcow2.zip))
(!url.toLowerCase().endsWith(qcow2.bz2))(!url.toLowerCase().endsWith(qcow2.gz))
(!url.toLowerCase().endsWith(ova))(!url.toLowerCase().endsWith(ova.zip))
(!url.toLowerCase().endsWith(ova.bz2))(!url.toLowerCase().endsWith(ova.gz))
(!url.toLowerCase().endsWith(tar))(!url.toLowerCase().endsWith(tar.zip))
(!url.toLowerCase().endsWith(tar.bz2))(!url.toLowerCase().endsWith(tar.gz))
(!url.toLowerCase().endsWith(vmdk))
(!url.toLowerCase().endsWith(vmdk.gz))
(!url.toLowerCase().endsWith(vmdk.zip))
(!url.toLowerCase().endsWith(vmdk.bz2))
(!url.toLowerCase().endsWith(img))
(!url.toLowerCase().endsWith(img.gz))
(!url.toLowerCase().endsWith(img.zip))
(!url.toLowerCase().endsWith(img.bz2))
(!url.toLowerCase().endsWith(raw))
(!url.toLowerCase().endsWith(raw.gz))
(!url.toLowerCase().endsWith(raw.bz2))
(!url.toLowerCase().endsWith(raw.zip))){
throw new InvalidParameterValueException(Please specify a
valid + format.toLowerCase());
}
if ((format.equalsIgnoreCase(vhd)
(!url.toLowerCase().endsWith(vhd)
!url.toLowerCase().endsWith(vhd.zip)
!url.toLowerCase().endsWith(vhd.bz2)
!url.toLowerCase().endsWith(vhd.gz)))
|| (format.equalsIgnoreCase(vhdx)
(!url.toLowerCase().endsWith(vhdx)
!url.toLowerCase().endsWith(vhdx.zip)
!url.toLowerCase().endsWith(vhdx.bz2)
!url.toLowerCase().endsWith(vhdx.gz)))
|| (format.equalsIgnoreCase(qcow2)
(!url.toLowerCase().endsWith(qcow2)
!url.toLowerCase().endsWith(qcow2.zip)
!url.toLowerCase().endsWith(qcow2.bz2)
!url.toLowerCase().endsWith(qcow2.gz)))
|| (format.equalsIgnoreCase(ova)
(!url.toLowerCase().endsWith(ova)
!url.toLowerCase().endsWith(ova.zip)
!url.toLowerCase().endsWith(ova.bz2)
!url.toLowerCase().endsWith(ova.gz)))
|| (format.equalsIgnoreCase(tar)
(!url.toLowerCase().endsWith(tar)
!url.toLowerCase().endsWith(tar.zip)
!url.toLowerCase().endsWith(tar.bz2)
!url.toLowerCase().endsWith(tar.gz)))
|| (format.equalsIgnoreCase(raw)
(!url.toLowerCase().endsWith(img)
!url.toLowerCase().endsWith(img.zip)
!url.toLowerCase().endsWith(img.bz2)
!url.toLowerCase().endsWith(img.gz)
!url.toLowerCase().endsWith(raw)
!url.toLowerCase().endsWith(raw.bz2)
!url.toLowerCase().endsWith(raw.zip)
!url.toLowerCase().endsWith(raw.gz)))
|| (format.equalsIgnoreCase(vmdk)
(!url.toLowerCase().endsWith(vmdk)
