When you are seeing errors early in the configure phase it is possible that you do not have a complete development environment. gcc alone is not enough.
Give this a whirl: yum groupinstall 'Development Tools' This is going to install many things but at least you wont be chasing down dependencies 1 by 1. Edward On Fri, Jan 15, 2010 at 4:17 PM, Todd Lipcon <t...@cloudera.com> wrote: > Are you starting from a clean tarball of the lzo stuff? > > Can you make sure your /tmp/ partition isn't full? > > There should be a config.log file hanging around somewhere also - perhaps > this would be helpful. > > -Todd > > On Fri, Jan 15, 2010 at 1:14 PM, Ted Yu <yuzhih...@gmail.com> wrote: > >> Todd: >> Thanks for the continued support. >> >> I installed lzo-devel: >> [r...@tyu-linux batchclient]# rpm -ivh >> /opt/kindsight/lzo-devel-2.02-2.el5.1.x86_64.rpm >> warning: /opt/kindsight/lzo-devel-2.02-2.el5.1.x86_64.rpm: Header V3 DSA >> signature: NOKEY, key ID 217521f6 >> Preparing... ########################################### >> [100%] >> 1:lzo-devel ########################################### >> [100%] >> >> But I still see: >> [exec] Try `/usr/bin/install --help' for more information. >> [exec] checking for gcc... gcc >> [exec] checking for C compiler default output file name... >> [exec] configure: error: C compiler cannot create executables >> >> >> On Fri, Jan 15, 2010 at 12:09 PM, Todd Lipcon <t...@cloudera.com> wrote: >> >> > Hi Ted, >> > >> > Did you also install liblzo-devel? >> > >> > Here are the packages I install for LZO: >> > >> > lzo-2.02-2.el5.1 >> > lzo-devel-2.02-2.el5.1 >> > >> > -Todd >> > >> > On Tue, Jan 12, 2010 at 4:28 PM, Ted Yu <yuzhih...@gmail.com> wrote: >> > >> > > Package gcc-4.1.2-46.el5_4.1.x86_64 already installed and latest >> version >> > > Nothing to do >> > > >> > > [r...@tyu-linux batchclient]# yum install gcc-c++ >> > > Loaded plugins: fastestmirror >> > > Loading mirror speeds from cached hostfile >> > > * base: mirrors.kernel.org >> > > * updates: updates.interworx.info >> > > * addons: mirrors.kernel.org >> > > * extras: mirrors.netdna.com >> > > Setting up Install Process >> > > Parsing package install arguments >> > > Package gcc-c++-4.1.2-46.el5_4.1.x86_64 already installed and latest >> > > version >> > > >> > > >> > > On Tue, Jan 12, 2010 at 2:32 PM, Todd Lipcon <t...@cloudera.com> >> wrote: >> > > >> > > > Make sure you have gcc, g++ installed? >> > > > >> > > > -Todd >> > > > >> > > > On Tue, Jan 12, 2010 at 10:57 AM, Ted Yu <yuzhih...@gmail.com> >> wrote: >> > > > >> > > > > I installed >> > > > > >> > > > > >> > > > >> > > >> > >> ftp://fr.rpmfind.net/linux/EPEL/5/x86_64/lzo-2.02-2.el5.1.x86_64.rpmyesterday >> > > > > . >> > > > > >> > > > > [r...@tyu-linux software]# rpm -e lzo >> > > > > [r...@tyu-linux software]# rpm -ivh >> > > ~rialto/lzo-2.02-2.el5.1.x86_64.rpm >> > > > > warning: /home/rialto/lzo-2.02-2.el5.1.x86_64.rpm: Header V3 DSA >> > > > signature: >> > > > > NOKEY, key ID 217521f6 >> > > > > Preparing... >> > ########################################### >> > > > > [100%] >> > > > > 1:lzo >> > ########################################### >> > > > > [100%] >> > > > > >> > > > > [ria...@tyu-linux java]$ ls -l /usr/lib64/liblzo2.so.2* >> > > > > lrwxrwxrwx 1 root root 16 Jan 12 10:29 /usr/lib64/liblzo2.so.2 >> -> >> > > > > liblzo2.so.2.0.0 >> > > > > -rwxr-xr-x 1 root root 123408 May 6 2007 >> > /usr/lib64/liblzo2.so.2.0.0 >> > > > > >> > > > > I still see this error: >> > > > > [exec] configure: error: C compiler cannot create executables >> > > > > [exec] See `config.log' for more details. >> > > > > >> > > > > BUILD FAILED >> > > > > /home/rialto/kevinweil-hadoop-lzo-916aeae/build.xml:243: exec >> > returned: >> > > > 77 >> > > > > >> > > > > >> > > > > On Tue, Jan 12, 2010 at 10:25 AM, Todd Lipcon <t...@cloudera.com> >> > > wrote: >> > > > > >> > > > > > Hi Ted, >> > > > > > >> > > > > > Please make sure you have version 2.02 of liblzo installed. >> There's >> > > an >> > > > > > earlier version in RHEL that doesn't fail properly at the >> configure >> > > > stage >> > > > > > and I've seen it produce the error you pasted below. >> > > > > > >> > > > > > -Todd >> > > > > > >> > > > > > On Tue, Jan 12, 2010 at 10:15 AM, Ted Yu <yuzhih...@gmail.com> >> > > wrote: >> > > > > > >> > > > > > > I followed >> > > http://code.google.com/p/hadoop-gpl-compression/wiki/FAQ >> > > > > > > >> > > > > > > Package gcc-c++-4.1.2-46.el5_4.1.x86_64 already installed and >> > > latest >> > > > > > > version >> > > > > > > Linux tyu-linux 2.6.18-128.2.1.el5 #1 SMP Tue Jul 14 06:36:37 >> EDT >> > > > 2009 >> > > > > > > x86_64 x86_64 x86_64 GNU/Linux >> > > > > > > >> > > > > > > For 32 bit build, I got: >> > > > > > > [exec] checking for C compiler default output file name... >> > > > > > > [exec] configure: error: C compiler cannot create >> executables >> > > > > > > [exec] See `config.log' for more details. >> > > > > > > >> > > > > > > BUILD FAILED >> > > > > > > /home/rialto/kevinweil-hadoop-lzo-916aeae/build.xml:243: exec >> > > > returned: >> > > > > > 77 >> > > > > > > >> > > > > > > For 64 bit build, I got: >> > > > > > > [exec] gcc -DHAVE_CONFIG_H -I. >> > > > > > > -I/home/rialto/kevinweil-hadoop-lzo-916aeae/src/native -I./impl >> > > > > > > -I/usr/java/default/include -I/usr/java/default/include/linux >> > > > > > > -I/home/rialto/kevinweil-hadoop-lzo-916aeae/src/native/impl >> > > > > > > -Isrc/com/hadoop/compression/lzo -g -Wall -fPIC -O2 -m64 -m64 >> -MT >> > > > > > > impl/lzo/LzoCompressor.lo -MD -MP -MF >> > > > impl/lzo/.deps/LzoCompressor.Tpo >> > > > > -c >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> /home/rialto/kevinweil-hadoop-lzo-916aeae/src/native/impl/lzo/LzoCompressor.c >> > > > > > > -fPIC -DPIC -o impl/lzo/.libs/LzoCompressor.o >> > > > > > > [exec] >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> /home/rialto/kevinweil-hadoop-lzo-916aeae/src/native/impl/lzo/LzoCompressor.c: >> > > > > > > In function >> > > 'Java_com_hadoop_compression_lzo_LzoCompressor_initIDs': >> > > > > > > [exec] >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> /home/rialto/kevinweil-hadoop-lzo-916aeae/src/native/impl/lzo/LzoCompressor.c:123: >> > > > > > > error: expected expression before ',' token >> > > > > > > [exec] make: *** [impl/lzo/LzoCompressor.lo] Error 1 >> > > > > > > >> > > > > > > BUILD FAILED >> > > > > > > /home/rialto/kevinweil-hadoop-lzo-916aeae/build.xml:251: exec >> > > > returned: >> > > > > 2 >> > > > > > > >> > > > > > > Has anybody seen the above ? >> > > > > > > >> > > > > > > Thanks >> > > > > > > >> > > > > > > On Mon, Jan 11, 2010 at 3:34 PM, Todd Lipcon < >> t...@cloudera.com> >> > > > > wrote: >> > > > > > > >> > > > > > > > Hi Ted, >> > > > > > > > >> > > > > > > > You need to install liblzo from EPEL: >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> http://fr.rpmfind.net/linux/RPM/Extras_Packages_for_Enterprise_Linux.html >> > > > > > > > >> > > > > > > > -Todd >> > > > > > > > >> > > > > > > > On Mon, Jan 11, 2010 at 3:21 PM, Ted Yu <yuzhih...@gmail.com >> > >> > > > wrote: >> > > > > > > > >> > > > > > > > > Can someone tell me how I can install liblzo ? >> > > > > > > > > >> > > > > > > > > [r...@tyu-linux lzo-2.03]# uname -a >> > > > > > > > > Linux tyu-linux 2.6.18-128.2.1.el5 #1 SMP Tue Jul 14 >> 06:36:37 >> > > EDT >> > > > > > 2009 >> > > > > > > > > x86_64 x86_64 x86_64 GNU/Linux >> > > > > > > > > [r...@tyu-linux lzo-2.03]# yum install liblzo-devel >> > > > > > > > > Loaded plugins: fastestmirror >> > > > > > > > > Loading mirror speeds from cached hostfile >> > > > > > > > > * base: mirrors.usc.edu >> > > > > > > > > * updates: mirror.san.fastserv.com >> > > > > > > > > * addons: centos.promopeddler.com >> > > > > > > > > * extras: mirrors.versaweb.com >> > > > > > > > > Setting up Install Process >> > > > > > > > > Parsing package install arguments >> > > > > > > > > No package liblzo-devel available. >> > > > > > > > > Nothing to do >> > > > > > > > > >> > > > > > > > > Thanks >> > > > > > > > > >> > > > > > > > > On Mon, Jan 11, 2010 at 12:45 PM, Steve Kuo < >> > > kuosen...@gmail.com >> > > > > >> > > > > > > wrote: >> > > > > > > > > >> > > > > > > > > > Ted, >> > > > > > > > > > >> > > > > > > > > > You may want to consider LZO compression, which allows >> > > > splitting >> > > > > > for >> > > > > > > a >> > > > > > > > > > comporessed file for Map jobs. On the other hand, gzip >> is >> > > not >> > > > > > > > > splittable. >> > > > > > > > > > >> > > > > > > > > > Check out these links. >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> http://www.cloudera.com/blog/2009/11/17/hadoop-at-twitter-part-1-splittable-lzo-compression/ >> > > > > > > > > > http://wiki.apache.org/hadoop/UsingLzoCompression >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > On Fri, Jan 8, 2010 at 1:13 PM, Ted Yu < >> > yuzhih...@gmail.com> >> > > > > > wrote: >> > > > > > > > > > >> > > > > > > > > > > The input file is in .gz format >> > > > > > > > > > > FYI >> > > > > > > > > > > >> > > > > > > > > > > On Fri, Jan 8, 2010 at 11:08 AM, Ted Yu < >> > > yuzhih...@gmail.com >> > > > > >> > > > > > > wrote: >> > > > > > > > > > > >> > > > > > > > > > > > My current project processes input file of size >> > 333302161 >> > > > > > bytes. >> > > > > > > > > > > > What I plan to do is to split the file into equal >> size >> > > > pieces >> > > > > > > (and >> > > > > > > > on >> > > > > > > > > > > blank >> > > > > > > > > > > > line boundary) to improve performance. >> > > > > > > > > > > > >> > > > > > > > > > > > I found 12 classes in 0.20.1 source code which >> > implement >> > > > > > > > InputSplit. >> > > > > > > > > > > > >> > > > > > > > > > > > If someone has written code similar to what I plan to >> > do, >> > > > > > please >> > > > > > > > > share >> > > > > > > > > > > some >> > > > > > > > > > > > hint. >> > > > > > > > > > > > >> > > > > > > > > > > > Thanks >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > On Fri, Jan 8, 2010 at 2:27 AM, Amogh Vasekar < >> > > > > > > am...@yahoo-inc.com >> > > > > > > > > >> > > > > > > > > > > wrote: >> > > > > > > > > > > > >> > > > > > > > > > > >> Hi, >> > > > > > > > > > > >> The deprecation is due to the new evolving mapreduce >> ( >> > > > > > > > > o.a.h.mapreduce >> > > > > > > > > > ) >> > > > > > > > > > > >> APIs. Old APIs are supported for available >> > > distributions. >> > > > > The >> > > > > > > > > > equivalent >> > > > > > > > > > > of >> > > > > > > > > > > >> TextInputFormat is available in new API : >> > > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/lib/input/TextInputFormat.html >> > > > > > > > > > > >> >> > > > > > > > > > > >> Thanks, >> > > > > > > > > > > >> Amogh >> > > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > > > > > >> On 1/8/10 3:47 AM, "Ted Yu" <yuzhih...@gmail.com> >> > > wrote: >> > > > > > > > > > > >> >> > > > > > > > > > > >> According to: >> > > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/TextInputFormat.html#isSplitable%28org.apache.hadoop.fs.FileSystem,%20org.apache.hadoop.fs.Path%29 >> > > > > > > > > > > >> >> > > > > > > > > > > >> isSplitable() is deprecated. >> > > > > > > > > > > >> >> > > > > > > > > > > >> Which method should I use to replace it ? >> > > > > > > > > > > >> >> > > > > > > > > > > >> Thanks >> > > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> >
