You're welcome, Marko. Glad to help you. On Thu, Jun 26, 2014 at 6:13 PM, Marko Loparic <[email protected]> wrote:
> Hi Jayy, > > Sorry, I had misunderstood your initial message (I though NodeImage > would give me a node but it gives me an image! duh...). I see now > clearly what I have to do. I will make a test as soon as I solve my > other issue (unrelated to libcloud). > > Thanks a lot for your help! > Marko > > On Thu, Jun 26, 2014 at 12:59 PM, Jayy Vis <[email protected]> wrote: > > You can launch as many instances as you want by specifying the count. And > > the your create_node snippet looks fine. > > > > Are you facing any problem? > > > > BTW, create_node prepares request to amazon RunInstances api using the > > arguments passed to it (image, size, ex_mincount, ex_maxcount). And the > > image object being passed to create_node is libcloud's NodeImage type. > > > > image = kwargs["image"] > > size = kwargs["size"] > > params = { > > 'Action': 'RunInstances', > > 'ImageId': *image.id <http://image.id>*, > > 'MinCount': str(kwargs.get('ex_mincount', '1')), > > 'MaxCount': str(kwargs.get('ex_maxcount', '1')), > > 'InstanceType': size.id > > } > > > > > > Best wishes, > > Jayy Vis | @jayyvis > > Co-founder, Pickle.io > > > > > > On Thu, Jun 26, 2014 at 3:42 PM, Marko Loparic <[email protected]> > > wrote: > > > >> Hello, > >> > >> Thanks a lot for all the answers! Each of them points to an > >> interesting solution for my case. Yes, I was using a t1.micro instance > >> indeed. > >> > >> Unfortunately I wasn't able to make tests yet (because of another > >> totally independent firewall issue). > >> > >> Do you know if I can launch several nodes at once using what Jayy Vis > >> proposed? I launch several machines at once and I suspect that > >> launching one by one would take a lot of time. I use > >> > >> nodes = conn.create_node(name, image=image, size=size, > >> ex_keyname=EC2_KEYNAME, > >> ex_mincount=count, ex_maxcount=count) > >> > >> where count is usually between 500 and 2000. From my understanding of > >> the code it seems that the create_node method for ec2 > >> (BaseEC2NodeDriver.create_method) uses an amazon RunInstances request > >> and is not based on NodeImage. > >> > >> Thanks again, > >> Marko > >> > >> On Wed, Jun 25, 2014 at 9:24 PM, Tomaz Muraus <[email protected]> wrote: > >> > Yeah, what others have said. > >> > > >> > By default, calling list_images in the EC2 driver without any filters > >> > returns a huge XML response. If lxml is not available (this feature is > >> only > >> > available in 0.15.0), parsing this response takes a while and > consumes a > >> > lot of memory. > >> > > >> > If you are not dynamically obtaining the image based on the name and > you > >> > already know the id, I would recommend you to use the approach Jayy > Vis > >> has > >> > suggested. > >> > > >> > On Wed, Jun 25, 2014 at 7:13 AM, Jayy Vis <[email protected]> wrote: > >> > > >> >> create_node api of libcloud just uses image.id > >> >> > >> >> So you can simply create image object directly as follows. > >> >> > >> >> Driver = get_driver(Provider.EC2_US_EAST) > >> >> conn = Driver(EC2_ACCESS_ID, EC2_SECRET) > >> >> > >> >> from libcloud.compute.base import NodeImage > >> >> > >> >> ec2_ami_id='ami-123' > >> >> > >> >> image=NodeImage(driver=conn, id=ec2_ami_id, name=ec2_ami_id, > extra=None) > >> >> > >> >> > >> >> > >> >> On Wed, Jun 25, 2014 at 3:36 AM, Lehman, Matthew J < > >> [email protected]> > >> >> wrote: > >> >> > >> >> > Do you guys have lxml installed? With the dev release it will use > lxml > >> >> for > >> >> > parsing if installed and that should help a bit. > >> >> > > >> >> > > >> >> > > >> >> > -----Original Message----- > >> >> > From: Joseph Hall [[email protected]<mailto:[email protected] > >] > >> >> > Sent: Tuesday, June 24, 2014 05:52 PM Eastern Standard Time > >> >> > To: [email protected] > >> >> > Subject: Re: list_images crashes (for ec2) > >> >> > > >> >> > > >> >> > Where are you actually running this code from? I have found in > recent > >> >> > months that, when running libcloud against EC2 on a micro instance, > >> >> > list_images() returns just too many images for the machine to > handle, > >> >> > and eventually the OOM killer destroys the process. > >> >> > > >> >> > On Tue, Jun 24, 2014 at 3:33 PM, Marko Loparic < > >> [email protected]> > >> >> > wrote: > >> >> > > Hello, > >> >> > > > >> >> > > The following script crashes when running on Amazon instance. > >> >> > > > >> >> > > ================================ > >> >> > > from libcloud.compute.types import Provider > >> >> > > from libcloud.compute.providers import get_driver > >> >> > > > >> >> > > EC2_ACCESS_ID = ... > >> >> > > EC2_SECRET = ... > >> >> > > Driver = get_driver(Provider.EC2_US_EAST) > >> >> > > conn = Driver(EC2_ACCESS_ID, EC2_SECRET) > >> >> > > > >> >> > > print "before" > >> >> > > conn.list_images() > >> >> > > print "after" > >> >> > > ================================ > >> >> > > > >> >> > > The first print appears and "Killed" is displayed before the > second > >> >> > print. > >> >> > > > >> >> > > I suspect it is an error in etree module (apparently the list of > >> >> > > available images is huge). > >> >> > > > >> >> > > I use list_images to get the python object of the desired image > >> >> > > > >> >> > > image = [i for i in conn.list_images() if i.id == > EC2_AMI][0] > >> >> > > > >> >> > > so that I can create a node with this image (create_node method) > >> >> > > > >> >> > > Do you know if there is a way to create a node without using > >> >> > > list_images or if there is another way to avoid this crash? > >> >> > > > >> >> > > I am using python 2.7.3 and libcloud 0.14.1. > >> >> > > > >> >> > > Thanks a lot in advance! > >> >> > > Marko > >> >> > > >> >> > > >> >> > > >> >> > -- > >> >> > "In order to create, you have to have the willingness, the desire > to > >> >> > be challenged, to be learning." -- Ferran Adria (speaking at > Harvard, > >> >> > 2011) > >> >> > > >> >> > >> >
