Yes - of course the different architecture means the existing image won't work! I've created https://issues.apache.org/jira/browse/HADOOP-2411 to track this issue. See this thread for how to take advantage of extra CPUs: http://www.mail-archive.com/hadoop-user@lucene.apache.org/msg02377.html.
Tom On 12/12/2007, Thibaut Britz <[EMAIL PROTECTED]> wrote: > > Hi Tom, > > I got the following error "Client.InvalidParameterValue: The requested > instance type does not agree with the architecture specified in the AMI > manifest." > During meantime, I built a private AMI from the 64 bit version of fedora > with hadoop version 14.4. > > What do I have to do to take advantages of the extra CPUS and the Ram (I > only increased the mapred.child.java.opts)? > > > My map/reduce code is comparable to wordcount (7413862 combine input pairs, > 122475 reduce input pairs) and doesn't make a lot of calculations, but still > takes over 30 seconds to complete, while the cpu is displayed to be idle > about 99% of the time in top. (If I don't collect pairs in the map function, > the job finished in 3 seconds (just to test how long it takes for my > RecordReader to deliver the key/value pairs)). Also increasing the mappers > dramatically reduces performances. (The above is measured with 2 mappers > only on one x1.large instance). Any ideas on what causes the performance to > be so low? > > > > > Tom White wrote: > > > > Hi Thibaut, > > > > Do you know why the existing hadoop images don't work with larger > > instance types? What's the error message you're getting? > > > > It should be relatively easy to change the launch-hadoop-cluster > > script to specify the instance type. Also, there's some work to be > > done to configure Hadoop on larger images to take advantage of the > > extra CPUs and memory. > > > > BTW I just published a 0.15.1 AMI. > > > > Tom > > > > On 11/12/2007, Thibaut Britz <[EMAIL PROTECTED]> wrote: > >> > >> Hi, > >> > >> The current public images only work on the smaller instances. > >> It would be very helpful (save me some time) if someone would be so kind > >> create or publish their hadoop image. > >> > >> Thibaut > >> > >> -- > >> View this message in context: > >> http://www.nabble.com/public-Amazon-EC2-hadoop-images-for-larger-instances-%28m1.large-and-m1.xlarge%29-tp14276807p14276807.html > >> Sent from the Hadoop Users mailing list archive at Nabble.com. > >> > >> > > > > > > -- > > Blog: http://problemsworthyofattack.blogspot.com/ > > > > > > -- > View this message in context: > http://www.nabble.com/public-Amazon-EC2-hadoop-images-for-larger-instances-%28m1.large-and-m1.xlarge%29-tp14276807p14296045.html > Sent from the Hadoop Users mailing list archive at Nabble.com. > > -- Blog: http://problemsworthyofattack.blogspot.com/
