I can reproduce the problem on the training cloud:
$ for i in `seq 1 100`; do euca-create-volume -z TorstenCloud -s1; done
...
VOLUME vol-52B905E31 creating2010-09-02T06:27:50.814Z
VOLUME vol-5F2906521 creating2010-09-02T06:27:51.142Z
VOLUME vol-598F062A
Make sure you start with a clean installation when you rerun the test.
Thanks!
--
eucalyptus create and delete volumes sometimes fail on lvm commands (POC + ENT
configs)
https://bugs.launchpad.net/bugs/590929
You received this bug notification because you are a member of Ubuntu
Server Team,
** Changed in: jaguarundi
Status: Triaged = In Progress
** Changed in: eucalyptus (Ubuntu Lucid)
Status: Triaged = In Progress
--
eucalyptus create and delete volumes sometimes fail on lvm commands (POC + ENT
configs)
https://bugs.launchpad.net/bugs/590929
You received this bug
I spent most of today testing this. My first package (ppa1) was missing
the euca_rootwrap configuration changes necessary to allow the new LVM
commands used by the upstream package. The ppa2 package I just uploaded
adds that and solves this problem.
I'm happy with it so far. It has solve the
I can reproduce the problem on the training cloud:
$ for i in `seq 1 100`; do euca-create-volume -z TorstenCloud -s1; done
...
VOLUME vol-52B905E31 creating2010-09-02T06:27:50.814Z
VOLUME vol-5F2906521 creating2010-09-02T06:27:51.142Z
VOLUME vol-598F062A
Make sure you start with a clean installation when you rerun the test.
Thanks!
--
eucalyptus create and delete volumes sometimes fail on lvm commands (POC + ENT
configs)
https://bugs.launchpad.net/bugs/590929
You received this bug notification because you are a member of Ubuntu
Bugs, which is
** Changed in: jaguarundi
Status: Triaged = In Progress
** Changed in: eucalyptus (Ubuntu Lucid)
Status: Triaged = In Progress
--
eucalyptus create and delete volumes sometimes fail on lvm commands (POC + ENT
configs)
https://bugs.launchpad.net/bugs/590929
You received this bug
I spent most of today testing this. My first package (ppa1) was missing
the euca_rootwrap configuration changes necessary to allow the new LVM
commands used by the upstream package. The ppa2 package I just uploaded
adds that and solves this problem.
I'm happy with it so far. It has solve the
I've applied another fix (revno 1236, 1.6.2 branch) that makes cleanup
of volumes more robust (1235 solves part of the problem). This has been
tested on our end.
--
eucalyptus create and delete volumes sometimes fail on lvm commands (POC + ENT
configs)
https://bugs.launchpad.net/bugs/590929
You
Marking fix-released in ubuntu/maverick, as this is fixed in Eucalyptus
2.0.
** Changed in: eucalyptus (Ubuntu)
Status: Confirmed = Fix Released
--
eucalyptus create and delete volumes sometimes fail on lvm commands (POC + ENT
configs)
https://bugs.launchpad.net/bugs/590929
You received
Assigning this to myself.
I'll get a package built and tested locally, and then pushed to lucid-
proposed.
:-Dustin
** Changed in: eucalyptus (Ubuntu Lucid)
Status: Confirmed = Triaged
** Changed in: eucalyptus (Ubuntu Lucid)
Assignee: (unassigned) = Dustin Kirkland (kirkland)
**
Does anyone on this bug have a current setup affect by this bug that
they could try upgrading to a candidate binary package that should fix
this problem?
All of my clouds are Maverick at this point. I'm hoping someone here
has a Lucid 10.04 cloud...
--
eucalyptus create and delete volumes
FYI, I just sent a build to the ~ubuntu-virt PPA. It should be building
soon. Need someone to sniff test this before we push it to proposed.
--
eucalyptus create and delete volumes sometimes fail on lvm commands (POC + ENT
configs)
https://bugs.launchpad.net/bugs/590929
You received this bug
I've applied another fix (revno 1236, 1.6.2 branch) that makes cleanup
of volumes more robust (1235 solves part of the problem). This has been
tested on our end.
--
eucalyptus create and delete volumes sometimes fail on lvm commands (POC + ENT
configs)
https://bugs.launchpad.net/bugs/590929
You
Marking fix-released in ubuntu/maverick, as this is fixed in Eucalyptus
2.0.
** Changed in: eucalyptus (Ubuntu)
Status: Confirmed = Fix Released
--
eucalyptus create and delete volumes sometimes fail on lvm commands (POC + ENT
configs)
https://bugs.launchpad.net/bugs/590929
You received
Assigning this to myself.
I'll get a package built and tested locally, and then pushed to lucid-
proposed.
:-Dustin
** Changed in: eucalyptus (Ubuntu Lucid)
Status: Confirmed = Triaged
** Changed in: eucalyptus (Ubuntu Lucid)
Assignee: (unassigned) = Dustin Kirkland (kirkland)
**
Does anyone on this bug have a current setup affect by this bug that
they could try upgrading to a candidate binary package that should fix
this problem?
All of my clouds are Maverick at this point. I'm hoping someone here
has a Lucid 10.04 cloud...
--
eucalyptus create and delete volumes
FYI, I just sent a build to the ~ubuntu-virt PPA. It should be building
soon. Need someone to sniff test this before we push it to proposed.
--
eucalyptus create and delete volumes sometimes fail on lvm commands (POC + ENT
configs)
https://bugs.launchpad.net/bugs/590929
You received this bug
Fix has been applied to the 1.6.2 branch (revno 1235). There are a
couple of issues here.
The first one has to do with concurrent creation of 100+ volumes in a
tight loop. Some creates were failing (fix has been backported from
2.0).
The second issue was that it is possible to cause a race in
Fix has been applied to the 1.6.2 branch (revno 1235). There are a
couple of issues here.
The first one has to do with concurrent creation of 100+ volumes in a
tight loop. Some creates were failing (fix has been backported from
2.0).
The second issue was that it is possible to cause a race in
Also note that any transient failures in deletion are reported correctly
to the user now, i.e., a subsequent describe request will list any
volumes that could not be deleted for any underlying back end reason.
After this fix, the system should no longer be in an inconsistent state
where the
** Changed in: jaguarundi
Importance: Undecided = High
** Changed in: jaguarundi
Milestone: None = sales-training
** Changed in: jaguarundi
Milestone: sales-training = jaguar-ga
--
eucalyptus create and delete volumes sometimes fail on lvm commands
This problem occurs on the all-in-one configuration.
May also occur on ENT config.
** Summary changed:
- eucalyptus create and delete volumes sometimes fail on lvm commands
+ eucalyptus create and delete volumes sometimes fail on lvm commands (POC +
ENT configs)
--
eucalyptus create and
There are some points that should be kept in mind:
* this error is (quite possibly) a result of a race; the tests we usually
conduct are stress tests -- in this case, we try to allocate a few hundreds of
volumes in one single sweep -- we are requesting volume creation faster than
volumes can
** Changed in: jaguarundi
Importance: Undecided = High
** Changed in: jaguarundi
Milestone: None = sales-training
** Changed in: jaguarundi
Milestone: sales-training = jaguar-ga
--
eucalyptus create and delete volumes sometimes fail on lvm commands
This problem occurs on the all-in-one configuration.
May also occur on ENT config.
** Summary changed:
- eucalyptus create and delete volumes sometimes fail on lvm commands
+ eucalyptus create and delete volumes sometimes fail on lvm commands (POC +
ENT configs)
--
eucalyptus create and
There are some points that should be kept in mind:
* this error is (quite possibly) a result of a race; the tests we usually
conduct are stress tests -- in this case, we try to allocate a few hundreds of
volumes in one single sweep -- we are requesting volume creation faster than
volumes can
If I lower N from 200 to 50, I have been able to successfully do this a
dozen times or so.
for i in `seq 1 50` ; do euca-create-volume -z cluster1 -s 1 ; done
for i in `euca-describe-volumes | awk '{print $2}'` ; do euca-delete-volume $i
; done
@eucalyptus developers: is there a known or
** Also affects: jaguarundi
Importance: Undecided
Status: New
** Changed in: jaguarundi
Status: New = Confirmed
--
eucalyptus create and delete volumes sometimes fail on lvm commands
https://bugs.launchpad.net/bugs/590929
You received this bug notification because you are a
I and Chris were able to repeat this by creating a lot of volumes batch
mode, pretty much like Dustin points above. We got a mix of losetup
errors (IIRC, a lock held), and various VG/LV; the result is an
inconsistent state: m loop devices, n VGs, and l LVs, with l n m.
--
eucalyptus create and
The system I'm working on here has a 127GB disk. It seems I'm able to
create 50 x 1GB volumes, and then I get nothing but this error:
Volume: Error while communicating with Storage Controller:
CreateStorageVolume:Your proposed upload exceeds the maximum allowed object
size.
--
eucalyptus
I have been unable to reproduce this issue against 2.0. Going to try
against 1.6.2 and report back.
@Dustin,
Volume: Error while communicating with Storage Controller:
CreateStorageVolume:Your proposed upload exceeds the maximum allowed
object size.
Web UI-configuration tab. Please increase the
Okay, I verified that cluster1.storage.maxtotalvolumesizeingb in the web
front end is greater than the size of volumes I'm trying to create.
I tried to create 100 x 1GB volumes, and 97 succeeded. However, I did
get 3 failures:
$ euca-describe-volumes | grep failed
VOLUME vol-5FBB065C 1
Marking it invalid against the latest dev branch (2.0), but I will try
to reproduce it against 1.6.2.
** Changed in: eucalyptus
Status: New = Invalid
--
eucalyptus create and delete volumes sometimes fail on lvm commands
https://bugs.launchpad.net/bugs/590929
You received this bug
I have also tried this with a sleep 1, in case we have a race condition
for i in `seq 1 200` ; do euca-create-volume -z cluster1 -s 1; sleep 1 ;
done
After creating a bunch (162) of volumes, I ended up getting:
Volume: Error while communicating with Storage Controller:
Against 1.6.2, I tried a test which involved creating and deleting 50
volumes in a tight loop, which passed successfully. With 100 volumes, I
was able to trigger a failure to clean up volumes. In addition, once the
max volume limit is reached and exceeded, occasionally, volumes are not
cleaned up
If I lower N from 200 to 50, I have been able to successfully do this a
dozen times or so.
for i in `seq 1 50` ; do euca-create-volume -z cluster1 -s 1 ; done
for i in `euca-describe-volumes | awk '{print $2}'` ; do euca-delete-volume $i
; done
@eucalyptus developers: is there a known or
** Also affects: jaguarundi
Importance: Undecided
Status: New
** Changed in: jaguarundi
Status: New = Confirmed
--
eucalyptus create and delete volumes sometimes fail on lvm commands
https://bugs.launchpad.net/bugs/590929
You received this bug notification because you are a
I and Chris were able to repeat this by creating a lot of volumes batch
mode, pretty much like Dustin points above. We got a mix of losetup
errors (IIRC, a lock held), and various VG/LV; the result is an
inconsistent state: m loop devices, n VGs, and l LVs, with l n m.
--
eucalyptus create and
The system I'm working on here has a 127GB disk. It seems I'm able to
create 50 x 1GB volumes, and then I get nothing but this error:
Volume: Error while communicating with Storage Controller:
CreateStorageVolume:Your proposed upload exceeds the maximum allowed object
size.
--
eucalyptus
I have been unable to reproduce this issue against 2.0. Going to try
against 1.6.2 and report back.
@Dustin,
Volume: Error while communicating with Storage Controller:
CreateStorageVolume:Your proposed upload exceeds the maximum allowed
object size.
Web UI-configuration tab. Please increase the
Okay, I verified that cluster1.storage.maxtotalvolumesizeingb in the web
front end is greater than the size of volumes I'm trying to create.
I tried to create 100 x 1GB volumes, and 97 succeeded. However, I did
get 3 failures:
$ euca-describe-volumes | grep failed
VOLUME vol-5FBB065C 1
Marking it invalid against the latest dev branch (2.0), but I will try
to reproduce it against 1.6.2.
** Changed in: eucalyptus
Status: New = Invalid
--
eucalyptus create and delete volumes sometimes fail on lvm commands
https://bugs.launchpad.net/bugs/590929
You received this bug
I have also tried this with a sleep 1, in case we have a race condition
for i in `seq 1 200` ; do euca-create-volume -z cluster1 -s 1; sleep 1 ;
done
After creating a bunch (162) of volumes, I ended up getting:
Volume: Error while communicating with Storage Controller:
Against 1.6.2, I tried a test which involved creating and deleting 50
volumes in a tight loop, which passed successfully. With 100 volumes, I
was able to trigger a failure to clean up volumes. In addition, once the
max volume limit is reached and exceeded, occasionally, volumes are not
cleaned up
Setting confirmed/Medium. Chris and I recreated this scenario on
different rigs.
** Changed in: eucalyptus (Ubuntu)
Importance: Undecided = Medium
** Changed in: eucalyptus (Ubuntu)
Status: New = Confirmed
--
eucalyptus create and delete volumes sometimes fail on lvm commands
This might actually be a problem with lvm, but even if it is caused by
lvm it should still be properly caught by eucalyptus so that if the
delete volume failure happens it will show up in euca-describe-volumes
** Also affects: eucalyptus (Ubuntu Lucid)
Importance: Undecided
Status: New
for the record: eucalyptus 1.6.2-0ubuntu30.1 (myself) and
1.6.2-0ubuntu30.2 (Chris, with his fixes for 30.2)
--
eucalyptus create and delete volumes sometimes fail on lvm commands
https://bugs.launchpad.net/bugs/590929
You received this bug notification because you are a member of Ubuntu
Server
This appears to be caused by eucalyptus not checking the return values
it gets.
--
eucalyptus create and delete volumes sometimes fail on lvm commands
https://bugs.launchpad.net/bugs/590929
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to
And to top it off lvm does not document its return codes, at least not
in the man pages anyway.
--
eucalyptus create and delete volumes sometimes fail on lvm commands
https://bugs.launchpad.net/bugs/590929
You received this bug notification because you are a member of Ubuntu
Server Team, which
It appears lvm always returns error code 5 on failure... for any kind of
failure. So I'm not sure exactly what we should do here to fix this
problem. Often the problem is actually just a can't get lock issue like
the following, but without having a useful error code we can't know when
to retry or
http://fedoraproject.org/wiki/LVM/liblvm
--
eucalyptus create and delete volumes sometimes fail on lvm commands
https://bugs.launchpad.net/bugs/590929
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to eucalyptus in ubuntu.
--
It appears the liblvm library is in newer lvm2 source which is in Debian
unstable. Whether converting eucalyptus to it is actually needed and/or
if it would be enough to help fix these sorts of issues is still
unknown. From a cursory look it appears it may be useful.
--
eucalyptus create and
It seems its not so much the not checking for the return code as not
doing the right thing afterwards. If it receives an error it appears to
just throw an exception and does not clean up after itself, at least for
the createLogicalVolume case.
--
eucalyptus create and delete volumes sometimes
Setting confirmed/Medium. Chris and I recreated this scenario on
different rigs.
** Changed in: eucalyptus (Ubuntu)
Importance: Undecided = Medium
** Changed in: eucalyptus (Ubuntu)
Status: New = Confirmed
--
eucalyptus create and delete volumes sometimes fail on lvm commands
This might actually be a problem with lvm, but even if it is caused by
lvm it should still be properly caught by eucalyptus so that if the
delete volume failure happens it will show up in euca-describe-volumes
** Also affects: eucalyptus (Ubuntu Lucid)
Importance: Undecided
Status: New
for the record: eucalyptus 1.6.2-0ubuntu30.1 (myself) and
1.6.2-0ubuntu30.2 (Chris, with his fixes for 30.2)
--
eucalyptus create and delete volumes sometimes fail on lvm commands
https://bugs.launchpad.net/bugs/590929
You received this bug notification because you are a member of Ubuntu
Bugs,
This appears to be caused by eucalyptus not checking the return values
it gets.
--
eucalyptus create and delete volumes sometimes fail on lvm commands
https://bugs.launchpad.net/bugs/590929
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
And to top it off lvm does not document its return codes, at least not
in the man pages anyway.
--
eucalyptus create and delete volumes sometimes fail on lvm commands
https://bugs.launchpad.net/bugs/590929
You received this bug notification because you are a member of Ubuntu
Bugs, which is
It appears lvm always returns error code 5 on failure... for any kind of
failure. So I'm not sure exactly what we should do here to fix this
problem. Often the problem is actually just a can't get lock issue like
the following, but without having a useful error code we can't know when
to retry or
http://fedoraproject.org/wiki/LVM/liblvm
--
eucalyptus create and delete volumes sometimes fail on lvm commands
https://bugs.launchpad.net/bugs/590929
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
--
ubuntu-bugs mailing list
It appears the liblvm library is in newer lvm2 source which is in Debian
unstable. Whether converting eucalyptus to it is actually needed and/or
if it would be enough to help fix these sorts of issues is still
unknown. From a cursory look it appears it may be useful.
--
eucalyptus create and
It seems its not so much the not checking for the return code as not
doing the right thing afterwards. If it receives an error it appears to
just throw an exception and does not clean up after itself, at least for
the createLogicalVolume case.
--
eucalyptus create and delete volumes sometimes
63 matches
Mail list logo
