Hey Ian, I know, and i really like the effort. I just doubt if spending effort on two things to achieve the same goal is worthwhile. If we want to run the simulator tests on Travis thats fine, but then don’t waste time and resources on running them on the jenkins.b.o. as well. If we focus our scarce resources in one direction we can do so much more i think.
That said, it is a job well done to get our stuff running on Travis. :D Cheers, Hugo On 1 sep. 2014, at 16:07, Ian Duffy <i...@ianduffy.ie> wrote: > Hi Hugo, > > Just to be clear this was just an investigation to see if people were > interested. > > Travis will give all forks of Cloudstack CI for free which is why it was > interesting. Just cause its enabled/there doesn't mean it has to be used > but it does give us another option should we wish to explore it. > > I made a commit to master this morning that would enable build jobs to > execute over there (all notification emails are disabled): > > https://travis-ci.org/apache/cloudstack/builds/34096969 > > The test results are mimicking that of jenkins, we get a failure on > test_09_expunge_vm. > > > On 1 September 2014 14:41, Hugo Trippaers <h...@trippaers.nl> wrote: > >> Sorry for the late reply, was away for some time.. >> >> >> >> On 21 aug. 2014, at 19:40, Edison Su <edison...@citrix.com> wrote: >> >>> >>> >>>> -----Original Message----- >>>> From: Trippie [mailto:trip...@gmail.com] On Behalf Of Hugo Trippaers >>>> Sent: Thursday, August 21, 2014 5:10 AM >>>> To: dev@cloudstack.apache.org >>>> Subject: Re: Integration Tests on TravisCI >>>> >>>> Hey guys, >>>> >>>> Just to make sure we are all on the same page on this. >>>> >>>> There are several initiatives to bolster our CI capabilities and some >> of them >>>> are overlapping. >>>> * There are ongoing discussion to push for a CI environment at the ASF >>>> * There are a number of companies donating hardware/cloud instances to >>>> beef up jenkins.buildacloud.org >>> Sorry for my ignorance, I don't know the situation yet. Can't wait for >> it to happen. >> >> Jenkins.buildacloud.org is already being supported by multiple companies >> and individuals, like Citrix, Schuberg Philis and the company Erik Weber is >> working for. David donated some space in his own private cloud as well. So >> we have quite decent resources already. We just need some more specialized >> equipment like F5, netscaler etc. >> >> >>> >>>> * Edison is doing CI builds on Amazon i believe >>>> * There is this TravisCI initiative >>>> >>>> While they are all very good, overlapping effort means we are wasting >>>> somebodies time somewhere. With a delicate a subject as CI we should try >>>> to avoid that i think. >>>> >>>> I'll happily stop working on the CI stuff in jenkins.buildacloud.org >> for example >>>> and focus my efforts on Travis. So i just want to know where we want to >> put >>>> our effort. >>> I don't think jenkins.buildacloud.org will go away. IMHO, the free >> TravisCI is good for simulator test, the hosted TravisCI >>> Is expensive. Nevertheless, we need hosted CI to test against real >> hardware, that's jenkins.buildacloud.orgcan help. >>> >>>> >>>> Cheers, >>>> >>>> Hugo >>>> >>>> >>>> >>>> >>>> On 21 aug. 2014, at 14:02, Ian Duffy <i...@ianduffy.ie> wrote: >>>> >>>>> Santosh, >>>>> >>>>> What command/flags did you use for the above test output? >>>>> >>>>> >>>>> On 14 August 2014 17:22, Santhosh Edukulla >>>>> <santhosh.eduku...@citrix.com> >>>>> wrote: >>>>> >>>>>> Ian, >>>>>> >>>>>> I just installed latest marvin after cloudstack build on master, and >>>>>> ran tests against the simulator from smoke folder, below are the >>>>>> results. All passed, except one, i believe that as well could be some >>>>>> global param or some config issue. I hope you want to install, use >>>>>> latest marvin and test suites from master. >>>>>> >>>>>> === TestName: test_DeployVmAntiAffinityGroup | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_deployvm_firstfit | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_deployvm_userconcentrated | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_deployvm_userdispersing | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_01_create_disk_offering | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_02_create_sparse_type_disk_offering | Status : >>>>>> SUCCESS === >>>>>> >>>>>> === TestName: test_04_create_fat_type_disk_offering | Status : >>>>>> SUCCESS === >>>>>> >>>>>> === TestName: test_02_edit_disk_offering | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_03_delete_disk_offering | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_UpdateConfigParamWithScope | Status : SUCCESS >>>> === >>>>>> >>>>>> === TestName: test_dedicateGuestVlanRange | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_01_create_iso | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_02_edit_iso | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_03_delete_iso | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_05_iso_permissions | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_nic_secondaryip_add_remove | Status : SUCCESS >>>> === >>>>>> >>>>>> === TestName: test_delete_account | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_public_ip_admin_account | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_public_ip_user_account | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_releaseIP | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_extendPhysicalNetworkVlan | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_UpdateStorageOverProvisioningFactor | Status : >>>>>> SUCCESS === >>>>>> >>>>>> === TestName: test_createPortablePublicIPAcquire | Status : SUCCESS >>>>>> === >>>>>> >>>>>> === TestName: test_createPortablePublicIPRange | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_01_primary_storage_nfs | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_privategw_acl | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_dedicatePublicIpRange | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_create_pvlan_network | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_createRegion | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_01_reset_vm_on_reboot | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_01_updatevolumedetail | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_03_restart_network_cleanup | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_05_router_basic | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_06_router_advanced | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_07_stop_router | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_08_start_router | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_09_reboot_router | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_01_sys_vm_start | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_02_sys_template_ready | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_01_create_service_offering | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_02_edit_service_offering | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_03_delete_service_offering | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_01_list_sec_storage_vm | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_02_list_cpvm_vm | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_01_create_template | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_02_edit_template | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_03_delete_template | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_05_template_permissions | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_07_list_public_templates | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_08_list_system_templates | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_advZoneVirtualRouter | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_deploy_vm | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_deploy_vm_multiple | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_01_stop_vm | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_02_start_vm | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_03_reboot_vm | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_06_destroy_vm | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_07_restore_vm | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_08_migrate_vm | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_09_expunge_vm | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_03_download_attached_volume | Status : SUCCESS >>>> === >>>>>> >>>>>> === TestName: test_04_delete_attached_volume | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_05_detach_volume | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_09_delete_detached_volume | Status : SUCCESS >>>> === >>>>>> >>>>>> === TestName: test_vpc_remote_access_vpn | Status : SUCCESS === >>>>>> >>>>>> === TestName: test_vpc_site2site_vpn | Status : SUCCESS === >>>>>> >>>>>> >>>>>> Thanks! >>>>>> Santhosh >>>>>> ________________________________________ >>>>>> From: Santhosh Edukulla >>>>>> Sent: Thursday, August 14, 2014 11:23 AM >>>>>> To: dev@cloudstack.apache.org >>>>>> Subject: RE: Integration Tests on TravisCI >>>>>> >>>>>> Few test suites under smoke/misc folder say test_vm_ha,test_vm_sync, >>>>>> test_deploy_vm have a dependency to run only on simulator and >>>>>> sequentially with no entries available in CS, these are written using >>>>>> simulator mocks, so if you run along with other test suites they will >> fail. >>>>>> >>>>>> with -w switch, nose will run all along with other suites under >>>>>> smoke, so, you can use noseexclude plugin and exclude them from >>>>>> normal run, and later run them in serially with out any dependencies. >>>>>> To exclude add the switch --exclude-dir="../smoke/misc" to nose >>>> command. >>>>>> >>>>>> Other should work. Where can we see the detailed logs for these runs? >>>>>> >>>>>> Regards, >>>>>> Santhosh >>>>>> ________________________________________ >>>>>> From: Ian Duffy [i...@ianduffy.ie] >>>>>> Sent: Thursday, August 14, 2014 9:42 AM >>>>>> To: CloudStack Dev >>>>>> Subject: Integration Tests on TravisCI >>>>>> >>>>>> Hi All, >>>>>> >>>>>> TL;DR: See https://github.com/imduffy15/Cloudstack-Clone specifically >>>>>> .travis.yml and tools/travis/* >>>>>> >>>>>> Myself, Sebastien and Rohit have been looking at executing >>>>>> Cloudstack's integration tests against the simulator using >> http://travis- >>>> ci.org. >>>>>> >>>>>> Doing this will enable us to execute the integration tests on all >>>>>> branches for free. It also means all community forks could make use >>>>>> of the travis-ci service too. >>>>>> >>>>>> >>>>>> Currently all tests are executing, there appears to be some tests >>>>>> failing on the simulator specifically: >>>>>> >>>>>> test_routers test_secondary_storage test_service_offerings test_ssvm >>>>>> test_templates test_vm_ha test_vm_life_cycle test_vm_sync >>>>>> test_volumes test_vpc_vpn >>>>>> >>>>>> Running these tests both locally and on travis-ci on the 4.4 branch >>>>>> I'm seeing failures. Anybody know why? >>>>>> >>>>>> Full console output of test execution available at >>>>>> https://travis-ci.org/imduffy15/Cloudstack-Clone/builds/32530691 >>>>>> >>>>>> For those interested in how all this is working: >>>>>> >>>>>> I have a fork of the Cloudstack source on my github: >>>>>> https://github.com/imduffy15/Cloudstack-Clone >>>>>> >>>>>> In the root of this repository you will find a .travis.yml file. This >>>>>> file executes a few different stages: >>>>>> >>>>>> before install: Installs all dependencies and build tools. >>>>>> install: compiles cloudstack >>>>>> before_script: starts up cloudstack and deploys the >>>>>> setup/dev/advanced.cfg >>>>>> script: executes the nosetests >>>>>> >>>>>> One limitation with travis-ci is that a job is not allowed to run for >>>>>> longer than 50mins. >>>>>> >>>>>> In order to get around this 3 separate environments are provisioned. >>>>>> They are described as below: >>>>>> >>>>>> Environment 1 runs test_affinity_groups test_deploy_vm >>>>>> test_deploy_vms_with_varied_deploymentplanners >>>>>> test_disk_offerings test_global_settings test_guest_vlan_range >>>>>> test_iso test_multipleips_per_nic test_network >>>>>> test_non_contigiousvlan >>>>>> >>>>>> Environment 2 runs: test_over_provisioning test_portable_publicip >>>>>> test_primary_storage test_privategw_acl test_public_ip_range >>>>>> test_pvlan test_regions test_reset_vm_on_reboot test_resource_detail" >>>>>> >>>>>> Environment 3 runs: test_routers test_secondary_storage >>>>>> test_service_offerings test_ssvm test_templates test_vm_ha >>>>>> test_vm_life_cycle test_vm_sync test_volumes test_vpc_vpn >>>>>> >>>>>> Thanks! >>>>>> >>>>>> Ian >> >>
