I recently received a private email from a Nutch user who had troubles
compiling the latest Hadoop native compression libraries due to the recent
addition of lzo2 compression support. It seems my previous method of compiling
for FreeBSD is somewhat outdated now, but the good news is there is a solution
and it allows us (FreeBSD users) to use lzo2 support!
Many of the same procedures will need to be done, since it still includes zlib
support. For the purpose of saving some space in this email you can refer to
this site for the archived message, including instructions on what needs to be
done for the first half of the operation:
http://www.mail-archive.com/[email protected]/msg06839.html.
We will need to install another port, this includes the library and headers for
lzo2 support on FreeBSD:
cd /usr/ports/archivers/lzo2/ && make install clean
After this is installed we need to copy the installed headers and library files
into other directories:
cp -R /usr/local/include/lzo /usr/include/
cp /usr/local/lib/liblzo2.* /usr/lib/
Now its time to run through all steps in my previous email (the one linked
above). You will need to copy one more file in addition to all the others, just
run this command:
cp src/org_apache_hadoop.h src/org/apache/hadoop/io/compress/lzo/
Now I haven't actually been able to test these new Hadoop native libraries in
production, but I'm personally expecting them to work fine. If you find any
problems, either during compilation or run-time use then let me know and we can
work out the bugs.
Sean Dean
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
