Hello,
Thanks a lot for all the answers! Each of them points to an
interesting solution for my case. Yes, I was using a t1.micro instance
indeed.
Unfortunately I wasn't able to make tests yet (because of another
totally independent firewall issue).
Do you know if I can launch several nodes at once using what Jayy Vis
proposed? I launch several machines at once and I suspect that
launching one by one would take a lot of time. I use
nodes = conn.create_node(name, image=image, size=size,
ex_keyname=EC2_KEYNAME,
ex_mincount=count, ex_maxcount=count)
where count is usually between 500 and 2000. From my understanding of
the code it seems that the create_node method for ec2
(BaseEC2NodeDriver.create_method) uses an amazon RunInstances request
and is not based on NodeImage.
Thanks again,
Marko
On Wed, Jun 25, 2014 at 9:24 PM, Tomaz Muraus <[email protected]> wrote:
> Yeah, what others have said.
>
> By default, calling list_images in the EC2 driver without any filters
> returns a huge XML response. If lxml is not available (this feature is only
> available in 0.15.0), parsing this response takes a while and consumes a
> lot of memory.
>
> If you are not dynamically obtaining the image based on the name and you
> already know the id, I would recommend you to use the approach Jayy Vis has
> suggested.
>
> On Wed, Jun 25, 2014 at 7:13 AM, Jayy Vis <[email protected]> wrote:
>
>> create_node api of libcloud just uses image.id
>>
>> So you can simply create image object directly as follows.
>>
>> Driver = get_driver(Provider.EC2_US_EAST)
>> conn = Driver(EC2_ACCESS_ID, EC2_SECRET)
>>
>> from libcloud.compute.base import NodeImage
>>
>> ec2_ami_id='ami-123'
>>
>> image=NodeImage(driver=conn, id=ec2_ami_id, name=ec2_ami_id, extra=None)
>>
>>
>>
>> On Wed, Jun 25, 2014 at 3:36 AM, Lehman, Matthew J <[email protected]>
>> wrote:
>>
>> > Do you guys have lxml installed? With the dev release it will use lxml
>> for
>> > parsing if installed and that should help a bit.
>> >
>> >
>> >
>> > -----Original Message-----
>> > From: Joseph Hall [[email protected]<mailto:[email protected]>]
>> > Sent: Tuesday, June 24, 2014 05:52 PM Eastern Standard Time
>> > To: [email protected]
>> > Subject: Re: list_images crashes (for ec2)
>> >
>> >
>> > Where are you actually running this code from? I have found in recent
>> > months that, when running libcloud against EC2 on a micro instance,
>> > list_images() returns just too many images for the machine to handle,
>> > and eventually the OOM killer destroys the process.
>> >
>> > On Tue, Jun 24, 2014 at 3:33 PM, Marko Loparic <[email protected]>
>> > wrote:
>> > > Hello,
>> > >
>> > > The following script crashes when running on Amazon instance.
>> > >
>> > > ================================
>> > > from libcloud.compute.types import Provider
>> > > from libcloud.compute.providers import get_driver
>> > >
>> > > EC2_ACCESS_ID = ...
>> > > EC2_SECRET = ...
>> > > Driver = get_driver(Provider.EC2_US_EAST)
>> > > conn = Driver(EC2_ACCESS_ID, EC2_SECRET)
>> > >
>> > > print "before"
>> > > conn.list_images()
>> > > print "after"
>> > > ================================
>> > >
>> > > The first print appears and "Killed" is displayed before the second
>> > print.
>> > >
>> > > I suspect it is an error in etree module (apparently the list of
>> > > available images is huge).
>> > >
>> > > I use list_images to get the python object of the desired image
>> > >
>> > > image = [i for i in conn.list_images() if i.id == EC2_AMI][0]
>> > >
>> > > so that I can create a node with this image (create_node method)
>> > >
>> > > Do you know if there is a way to create a node without using
>> > > list_images or if there is another way to avoid this crash?
>> > >
>> > > I am using python 2.7.3 and libcloud 0.14.1.
>> > >
>> > > Thanks a lot in advance!
>> > > Marko
>> >
>> >
>> >
>> > --
>> > "In order to create, you have to have the willingness, the desire to
>> > be challenged, to be learning." -- Ferran Adria (speaking at Harvard,
>> > 2011)
>> >
>>
