I have been using whirr call simple scripts using whirr run-script, but I have to say it's running way too slow to be usable.
Running parallel-ssh to call a simple script on a 2-node EC2 cluster takes about 7 seconds. Running the same script (more or less) takes more than 1min 30 secs. I know that's comparing I'm apples with comfy chairs: Whirr is making EC2 calls to check regions,zones, available hosts, ... but all those EC2 api calls (I have counted 17, see attached log file) come with a price (well, with time). Are the EC2 calls really necessary? Maybe we should skip them alltogether (look in the instances file) or provide caching cache the EC2 results... WDYT? Karel -- Karel Vervaeke http://outerthought.org/ Open Source Content Applications Makers of Kauri, Daisy CMS and Lily
2011-09-26 18:13:18,083 DEBUG [org.apache.whirr.service.ComputeCache] (main) creating new ComputeServiceContext org.apache.whirr.service.ComputeCache$Key@6dc1e027 2011-09-26 18:13:20,118 DEBUG [jclouds.wire] (i/o thread 0) >> "Action=DescribeRegions&Signature=XWUZxf6klRFXGinkNAoq%2B2tTzR1ELDee8NxbeVRZ%2B0w%3D&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-09-26T16%3A14%3A19.648Z&Version=2010-06-15&AWSAccessKeyId=AKIAJSBJS6H3BHO667IA" 2011-09-26 18:13:36,719 DEBUG [jclouds.wire] (i/o thread 0) >> "Action=DescribeAvailabilityZones&Signature=Ib78ZsGAdfe0Q7FUIVrPwCGwzs3VmGDYMASGjIdsWDI%3D&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-09-26T16%3A14%3A36.717Z&Version=2010-06-15&AWSAccessKeyId=AKIAJSBJS6H3BHO667IA" 2011-09-26 18:13:49,292 DEBUG [jclouds.wire] (i/o thread 0) >> "Action=DescribeAvailabilityZones&Signature=NYE6g4YHemxivAvumbqmCboyFLRJejh57%2Bey1QZYy5c%3D&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-09-26T16%3A14%3A49.283Z&Version=2010-06-15&AWSAccessKeyId=AKIAJSBJS6H3BHO667IA" 2011-09-26 18:14:05,291 DEBUG [jclouds.wire] (i/o thread 0) >> "Action=DescribeAvailabilityZones&Signature=StcsAWuyZ7wsGaz%2B80BNJ9KySQ%2FX5bV3L9koHb9NSSg%3D&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-09-26T16%3A15%3A05.286Z&Version=2010-06-15&AWSAccessKeyId=AKIAJSBJS6H3BHO667IA" 2011-09-26 18:14:05,664 DEBUG [jclouds.wire] (i/o thread 0) >> "Action=DescribeAvailabilityZones&Signature=7fR5ybALQlrgvVdPNPq80Y%2BMFUc60JWnyhokpU6u3Ew%3D&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-09-26T16%3A15%3A05.659Z&Version=2010-06-15&AWSAccessKeyId=AKIAJSBJS6H3BHO667IA" 2011-09-26 18:14:17,773 DEBUG [jclouds.wire] (i/o thread 0) << "<?xml version="1.0" encoding="UTF-8"?>[\n]" 2011-09-26 18:14:17,792 DEBUG [jclouds.wire] (i/o thread 0) >> "Action=DescribeAvailabilityZones&Signature=J0lsbqc7Pl78vDfkq7k2pNBL%2F41QGOohNtljlY%2BF87c%3D&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-09-26T16%3A15%3A17.787Z&Version=2010-06-15&AWSAccessKeyId=AKIAJSBJS6H3BHO667IA" 2011-09-26 18:14:30,552 DEBUG [org.apache.whirr.service.ComputeCache] (main) created new ComputeServiceContext [id=aws-ec2, endpoint=https://ec2.us-east-1.amazonaws.com, apiVersion=2010-06-15, identity=AKIAJSBJS6H3BHO667IA, iso3166Codes=[US-VA, US-CA, IE, SG, JP-13]] 2011-09-26 18:14:30,639 DEBUG [jclouds.wire] (i/o thread 0) >> "Action=DescribeSpotInstanceRequests&Signature=Ur%2F11fSdUg2%2FWBmeQCE%2F5cxCTRz2tveRAB84DOZV%2F%2Bs%3D&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-09-26T16%3A15%3A30.637Z&Version=2011-05-15&AWSAccessKeyId=AKIAJSBJS6H3BHO667IA" 2011-09-26 18:14:30,646 DEBUG [jclouds.wire] (i/o thread 1) >> "Action=DescribeSpotInstanceRequests&Signature=fvkbJfzlRvr2qGuUerwjKS9XMzzaAHZdG%2FkP7uSe00o%3D&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-09-26T16%3A15%3A30.644Z&Version=2011-05-15&AWSAccessKeyId=AKIAJSBJS6H3BHO667IA" 2011-09-26 18:14:30,652 DEBUG [jclouds.wire] (i/o thread 2) >> "Action=DescribeSpotInstanceRequests&Signature=Tjco0MoY%2F%2FpYZ%2FkZoOmRqrjCvaqnYBtwTxh%2B3%2BsZ35M%3D&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-09-26T16%3A15%3A30.650Z&Version=2011-05-15&AWSAccessKeyId=AKIAJSBJS6H3BHO667IA" 2011-09-26 18:14:30,658 DEBUG [jclouds.wire] (i/o thread 3) >> "Action=DescribeSpotInstanceRequests&Signature=kQloJBYan073OJrC3ip9yU59V3VXh22hrommfZTWYss%3D&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-09-26T16%3A15%3A30.657Z&Version=2011-05-15&AWSAccessKeyId=AKIAJSBJS6H3BHO667IA" 2011-09-26 18:14:30,891 DEBUG [jclouds.wire] (i/o thread 1) >> "Action=DescribeSpotInstanceRequests&Signature=AWu9uArIJdw9I8Syn9idwA%2B%2F%2FQ7U3%2Fj%2B35twORi73rg%3D&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-09-26T16%3A15%3A30.886Z&Version=2011-05-15&AWSAccessKeyId=AKIAJSBJS6H3BHO667IA" 2011-09-26 18:14:42,471 DEBUG [jclouds.wire] (i/o thread 3) >> "Action=DescribeInstances&Signature=6YIQ11m%2BT%2FRxNpf%2B6ufSuyiDl319jvNAzf16jBVcvPw%3D&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-09-26T16%3A15%3A42.470Z&Version=2011-05-15&AWSAccessKeyId=AKIAJSBJS6H3BHO667IA" 2011-09-26 18:14:42,475 DEBUG [jclouds.wire] (i/o thread 2) >> "Action=DescribeInstances&Signature=PUBUIE1KFzrnIoUNDjSuEzkZiDHqs%2BbbWtmKXtbdYPY%3D&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-09-26T16%3A15%3A42.473Z&Version=2011-05-15&AWSAccessKeyId=AKIAJSBJS6H3BHO667IA" 2011-09-26 18:14:42,477 DEBUG [jclouds.wire] (i/o thread 0) >> "Action=DescribeInstances&Signature=mWrKoqBuVa%2Ff2hdzzLh41SFl8ct0ZSWB0tXJ1hWhD2Y%3D&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-09-26T16%3A15%3A42.475Z&Version=2011-05-15&AWSAccessKeyId=AKIAJSBJS6H3BHO667IA" 2011-09-26 18:14:42,483 DEBUG [jclouds.wire] (i/o thread 1) >> "Action=DescribeInstances&Signature=GxbdfttmTyM6EdNqSO38IjmD27SSpy8mPIuDOx3Nt8k%3D&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-09-26T16%3A15%3A42.482Z&Version=2011-05-15&AWSAccessKeyId=AKIAJSBJS6H3BHO667IA" 2011-09-26 18:14:42,489 DEBUG [jclouds.wire] (i/o thread 4) >> "Action=DescribeInstances&Signature=Oe8nnKdpFnrcVs6%2BS1nAm0MkVxSkrBFfU5cavF5DdtQ%3D&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-09-26T16%3A15%3A42.488Z&Version=2011-05-15&AWSAccessKeyId=AKIAJSBJS6H3BHO667IA" 2011-09-26 18:14:43,775 DEBUG [jclouds.wire] (i/o thread 2) >> "Action=DescribeImages&ImageId.1=ami-da0cf8b3&Signature=yd%2B3MEnI465TMktFxJpqZadJpmR9GuYi1Wso83qKYUw%3D&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-09-26T16%3A15%3A43.774Z&Version=2011-05-15&AWSAccessKeyId=AKIAJSBJS6H3BHO667IA" 2011-09-26 18:14:43,960 DEBUG [jclouds.compute] (user thread 2) >> blocking on socket [address=107.20.15.86, port=22] for 600000 seconds 2011-09-26 18:14:43,962 DEBUG [jclouds.compute] (user thread 1) >> blocking on socket [address=50.19.29.41, port=22] for 600000 seconds 2011-09-26 18:14:44,053 DEBUG [jclouds.compute] (user thread 2) << socket [address=107.20.15.86, port=22] opened 2011-09-26 18:14:44,054 DEBUG [jclouds.compute] (user thread 1) << socket [address=50.19.29.41, port=22] opened 2011-09-26 18:14:45,695 DEBUG [jclouds.compute] (user thread 1) >> running [#!/bin/bash 2011-09-26 18:14:47,139 DEBUG [org.apache.whirr.service.ComputeCache] (Thread-1) closing ComputeServiceContext [id=aws-ec2, endpoint=https://ec2.us-east-1.amazonaws.com, apiVersion=2010-06-15, identity=AKIAJSBJS6H3BHO667IA, iso3166Codes=[US-VA, US-CA, IE, SG, JP-13]]
