Re: [openstack-dev] [third-party-ci]Tests randomly failing because of lvm

2015-08-20 Thread Eduard Matei
Hi, In case someone is still interested, 32768 is the fix. The number of PEs must be multiple of 1024 (not sure why). Eduard __ OpenStack Development Mailing List (not for usage questions) Unsubscribe:

Re: [openstack-dev] [third-party-ci]Tests randomly failing because of lvm

2015-08-18 Thread Eduard Matei
Hi, Further investigation shows there might be a relation between a test failing because it can't upload an image (most likely because there's not enough space left) and another test failing because lvm doesn't have enough extents - at the same time (due to tests running on two workers). So if

Re: [openstack-dev] [third-party-ci]Tests randomly failing because of lvm

2015-08-18 Thread Eduard Matei
Hi, Might be because volume backing size of 32G results in 8191 PE instead of 8192, so last volume create attempt sees only 1023 free extents. Will retry with 32768M Eduard __ OpenStack Development Mailing List (not for

Re: [openstack-dev] [third-party-ci]Tests randomly failing because of lvm

2015-08-17 Thread Eduard Matei
Hi, I can't find any way to change the image size of the vm used for testing (it's been created using nodepool-dib and managed by nodepoold). Anyway, tests are still (randomly) failing with lvm error (Volume group stack-volumes-lvmdriver-1 has insufficient free space (1023 extents): 1024

Re: [openstack-dev] [third-party-ci]Tests randomly failing because of lvm

2015-08-17 Thread Andrey Pavlov
glance log - 2015-08-17 03:13:42.938 2312 INFO swiftclient [req-463d065c-8c26-4be2-8e30-5f1d83e4a6b8 2c757dc5add24d02b867436d82db5e99 98a460b55a544902b4bfbb104e8fae7f - - -] REQ: curl -i http://127.0.0.1:8080/v1/AUTH_93bdc098999d400b9838abb51a7a8126/glance/32be6815-3571-48e8-b664-7902613ffd04 -X

Re: [openstack-dev] [third-party-ci]Tests randomly failing because of lvm

2015-08-17 Thread Eduard Matei
Thanks, I didn't see that errror, seems to be caused by swift: 2015-08-17 03:13:43.940 2312 ERROR glance.api.v1.upload_utils BackendException: Failed to add object to Swift. I'll investigate further. But what about the lvmdriver-1 error? Isn't that related. Thanks, Eduard

Re: [openstack-dev] [third-party-ci]Tests randomly failing because of lvm

2015-08-17 Thread Eduard Matei
Thanks, I will try to increase the size of the dsvm disk, maybe that will help. No idea about the image size. The problem is that not all test runs fail, so it's not easy to trace. Eduard __ OpenStack Development Mailing

[openstack-dev] [third-party-ci]Tests randomly failing because of lvm

2015-08-17 Thread Eduard Matei
Hi, I'm working on getting our CI working again, and i left it running over the weekend. Of the 50 jobs it ran. 6 of them failed because of different tests failed, but all with the same cause tempest.exceptions.VolumeBuildErrorException Looking in c-vol log i see Volume group

Re: [openstack-dev] [third-party-ci]Tests randomly failing because of lvm

2015-08-17 Thread Abhishek Shrivastava
Hi Eduard, Can you please send the snippet of failed tests along with their error cause from your Build Logs. On Mon, Aug 17, 2015 at 12:02 PM, Eduard Matei eduard.ma...@cloudfounders.com wrote: Hi, I'm working on getting our CI working again, and i left it running over the weekend. Of

Re: [openstack-dev] [third-party-ci]Tests randomly failing because of lvm

2015-08-17 Thread Abhishek Shrivastava
Hi Eduard, This is what I gathered the cause of the test failure: 2015-08-17 03:13:44.239 ERROR oslo_messaging.rpc.dispatcher [req-95c1bc0f-e333-493b-9730-cfba9c3dfd9a tempest-VolumesV1ActionsTest-418947994] Exception during message handling: 500 Internal Server Error: Failed to upload image

Re: [openstack-dev] [third-party-ci]Tests randomly failing because of lvm

2015-08-17 Thread Eduard Matei
Hi, Not sure what snippet you mean but here are all the logs for the failed test run: http://packages.cloudfounders.com/ci_logs/01/213501/3/check/dsvm-tempest-full/37aa6fe/ (console.html and c-vol.txt show more details). Thanks, Eduard